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FOREWORD 


It is a privilege for me to write the foreword for a book which 
initiated me to the subject. 

Writing a good textbook is always a challenge for any author. 
This challenge to write a book on a developing subject is still more 
demanding. Professor Hadley crossed all such hurdles when he wrote 
this book on Linear Programming. He put in it almost everything 
that was known about Linear Programming at that time. This was 
done in such a systematic way that the book is used even today by all 
those who know the intricacies of Linear Programming. The 
notations introduced in this book are being currently followed and 
referred to in most research papers. 

Inspite of the many interesting developments that have taken 
place since this book was first written, I am sure that this book will 
serve as a text book for many more years to come and continue to 
inspire the research workers to unfold the hidden aspects of Linear 
Programming. 

M/s Narosa Publishing House deserve all praise for publishing 
this edition of such an important book. 


S.K. Gupta 
Indian Institute of Technology 
Kanpur 208016, UP, India 
September 18, 1987 


PREFACE 


Only a little over a decade has passed since George Dantzig formulated 
the general linear programming problem and developed the simplex 
method for its solution. In this period, the growth of interest in, and the 
use of, linear programming has been remarkable. Rarely indeed has a new 
mathematical technique found such a wide range of practical applications, 
and simultaneously received so thorough a theoretical development in such 
a short period of time. The extensive interest in linear programming which 
has arisen has brought with it the need for texts at different levels of dif- 
ficulty, suitable for readers of widely varying backgrounds and mathe- 
matical maturity. The present work is intended for those who desire to 
study the subject in some depth and detail. It attempts to provide a 
fairly rigorous and complete development of the theoretical and compu- 
tational aspects of linear programming as well as a discussion of a number 
of practical applications. 

Chapter 1 introduces the general linear programming problem and 
exhibits a series of graphical examples. Chapter 2 covers the mathematical 
background needed. In Chapter 3 the fundamental theoretical results 
required for the simplex method are derived. Chapter 4 provides a de- 
tailed development of the computational procedure of the simplex method. 
The two-phase technique is introduced in Chapter 5, which also includes 
a discussion of the solutions and requirements spaces. Chapter 6 presents 
Charnes’ perturbation technique and the generalized simplex method for 
resolving the degeneracy problem. The revised simplex method is covered 
in Chapter 7. Chapter 8 is devoted to duality; included in this chapter 
are the dual simplex algorithm and the primal-dual algorithm. The 
solution of transportation problems is the concern of Chapter 9. A novel 
approach is used to derive the transportation algorithm from the simplex 
method, Generalized transportation problems are also covered in this 
chapter. . Chapter 10 discusses network flow problems, the primal-dual 
algorithm for transportation problems, assignment problems, and the 
transhipment problem. Chapter 11 treats a number of special topics, 
such as sensitivity analysis, treatment of upper bounds for the general 
linear programming problem, the primal-dual algorithm for capacitated 
transportation problems, the decomposition principle, and the relationships 
between linear programming and zero-sum two-person games. The ap- 
plication of linear programming to practical problems in industry is dis- 
cussed in Chapter 12, and applications to economic theory are considered 
in Chapter 13. 

The level of presentation in this book assumes that the reader has a 
familiarity with certain elementary topics in linear algebra (including 

v 


vi PREFACE 


convex sets). 'The necessary background material is reviewed in Chapter 2. 
However, tHe reader who has had no previous introduction to the material 
may find this review too abbreviated. It is suggested that such readers 
study coneurrently the author's Linear Algebra, which covers in detail the 
material needed. (Incidentally, the notation used is eonsistent throughout 
both volumes.) As ап aid to those who are simultaneously attempting to 
gain some knowledge of linear algebra, the present book develops its 
major themes in considerable detail, especially in Chapters 3, 4, 5, 6. It 
might be pointed out that a knowledge of convex sets in n dimensions is 
not essential for reading the text. The sections dealing with these topics 
can be omitted without loss of continuity. 

Although the text is fairly complete, there are several topies associated 
with linear programming which are not to be found here. Two such 
topies, namely, linear programming problems some of whose parameters 
may be random variables and the solution of the general linear program- 
ming problem in integers, are considered by the author to be special cases 
of nonlinear programming problems and are discussed in a separate volume 
entitled Nonlinear and Dynamic Programming. No account is given of the 
use of analog computers to sol¥e linear programming problems. Some ma- 
terial in this subject was included in an original version of the manuscript, 
but was dropped because it was felt to be a diversion from the main theme, 
of interest only to a small number of readers. General considerations 
regarding the solution of linear programming problems on digital com- 
puters are examined in the text, but no attempt is made to describe 
coding procedures in detail since the method chosen depends too much on 
the characteristics of the computer to be employed. Similarly, no detailed 
description of the linear programming codes available for various com- 
puters is given since these would be almost immediately out of date. 

The text, contains sufficient material for a two-semester course in linear 
programming although it can easily be used for a one-semester or one- 
quarter course in the subject. For example, the author has taught a 
one-semester graduate course at MIT devoted to the theory of linear 
programming. The students entering the course had no background in 
linear algebra, and hence about the first seven weeks were devoted to 
covering selected topics from the first six chapters of the author’s Linear 
Algebra. For the remainder of the semester the material covered consisted 
of Chapters 3, 4, 5 (through Section 5-4), 6 (through Section 6-6), 7, 
8 (through Section 8-7), and 9 (through Section 9-12) of the present 
volume. At the University of Chicago, the author has taught a one- 
quarter course to graduate students who had a course in linear algebra. 
After a brief review (about two weeks) of linear algebra and some discus- 
Sion of convex sets, the above-mentioned material, the first four sections 
of Chapter 11, and most of Chapter 12 of this book were covered in the 
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remainder of the quarter. It should be pointed out that this quarter course 
normally required about twelve hours per week of work outside of class. 

The present volume should also serve as a supplementary text for courses 
in mathematical economies, engineering mathematics, operations research, 
or other courses which attempt to provide a serious treatment of linear 
programming. Finally, because of its completeness, it should be useful as 
a reference and as a text for self-study. 

At the end of each chapter there will be found a collection of problems 
for solution. Some of these problems emphasize numerical techniques, 
while others concentrate on theoretical points. Within each of these two 
classes, there is a considerable range of difficulty. The author considers 
the problems to be very important, and anyone studying this work should, 
at the very least, read all the problems. When linear programming prob- 
lems are solved by hand, it is a good idea to use a desk calculator if at all 
possible, or at least a slide rule. In this way, the numbers computed will 
be obtained as decimals. Of course, if a large-scale digital computer is 
used, the answers obtained will also be in decimal form. It does not seem 
sound to get into the habit of solving problems by means of fractions 
instead of decimal numbers, since fractions become impossibly cumbersome 
unless the original coefficients are small integers. For this reason, most of 
the tableaux in the text are presented in decimal form, even though they 
could have been expressed more simply and accurately as fractions. 
When the decimal numbers given are not exact, each element in the 
tableaux is expressed with roughly the same relative error. The same 
number of decimal places does not appear in each element. 

For their helpful suggestions, the author wishes to express his apprecia- 
tion to Professors H. Houthakker and H. Wagner, who read an early 
version of the manuscript, and to Professors R. Dorfman and T. M. 
Whitin, who read a later version. The author is also indebted to one of 
his students, M. A. Simonnard, whose thesis laid the foundations for the 
method of development used in the initial sections of Chapter 9. Jackson E. 
Morris supplied most of the quotations which appear at the beginning of 
each chapter. The School of Industrial Management at the Massachusetts 
Institute of Technology generously provided secretarial assistance for 


typing the manuscript. 
G.H. 
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CHAPTER 1 
INTRODUCTION 


<“, , . Since a crooked figure may 

Attest in little place a million; 

Then let us, ciphers to this great account, 
On your imaginary forces work,” 


Shakespeare—Henry V. 


1-1 Optimization problems. Problems which seek to maximize or 
minimize à numerical function of a number of variables (or functions), 
with the variables (functions) subject to certain constraints, form a general 
class which may be called optimization problems. 

Many optimization problems were first encountered in the physical 
sciences and geometry. The quest for solutions led to applications of the 
differential calculus and to the development of the calculus of variations. 
These classical optimization techniques have been known for over 150 
years. They have been applied with considerable success to the solution 
of many problems in the physical sciences and engineering. Later, the 
differential calculus was found to be very useful in economics, especially 
in developing the important results in the classical theory of production 
and consumption . 

In the last ten or fifteen years, many new and important optimization 
problems have emerged in the field of economics and have received a great 
deal of attention. Asa class, these problems may be referred to as program- 
ming problems. They are of so much interest because of their applicability 
to practical problems in government, military and industrial operations, 
as well as to problems in economie theory. In general, classical optimiza- 
tion techniques have been found to be of little assistance in solving these 
programming problems. "Therefore, new methods had to be developed. 
In this book, we shall treat only a special but very important class of 
programming problems known as linear programming problems. We shall 
be concerned with the theory of linear programming, with numerical 
techniques for solving such problems, and with applications of the theory. 


1-2 Programming problems. Broadly speaking, programming prob- 
lems deal with determining optimal allocations of limited resources to 
meet given objectives; more specifically, they deal with situations where 
a number of resources, such as men, materials, machines, and land, are 
available, and are to be combined to yield one or more products. There 
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are, however, certain restrictions on all or some of the following broad 
categories, i.e.: on the total amount of each resource available, on the 
quantity of each product made, on the quality of each product. Even with- 
in these restrictions there will exist many feasible allocations. Out of all 
permissible allocations of resources, it is desired to find the one or ones 
which maximize or minimize some numerical quantity, such as profit or cost. 

The actual conversion of resources to products may be a simple mixing 
operation, such as mixing raw stock gasolines to form various motor fuels, 
or a complicated production process involving many types of machines 
and operations. In certain cases, the resources and products can be 
identical. For example: We may be interested in finding the cheapest way 
of transporting a product from a number of origins to a number of desti- 
nations. 

Linear programming deals with that class of programming problems for 
which all relations among the variables are linear. The relations must be 
linear both in the constraints and in the function to be optimized. 


1-3 An example. Let us consider a shop with three types of machines, 
A, B, and C, which can turn out four products, 1, 2, 3, 4. Any one of the 
products has to undergo some operation on each of the three types of 
machines (lathes, drills, and milling machines, for example). We shall 
assume that the production is continuous, and that each product must 
first go on machine type A, then B, and finally C. Furthermore, we shall 
assume that the time required for adjusting the setup of each machine 


to a different operation, when production shifts from one product to : 


another, is negligible. Table 1-1 shows: (1) the hours required on each 
machine type per unit of each product; (2) the total available machine 
hours per week; (3) the profit realized on the sale of one unit of any one 
of the products. It is assumed that the profit is directly proportional to 
the number of units sold. We wish to determine the weekly output for 
each product in order to maximize profits. 

Examination of Table 1-1 shows that the item with the highest unit 
profit requires a considerable amount of time on machines A and C; the 
product with the second-best unit profit requires relatively little time on 
machine A and slightly less time on machine C than the item with the 
highest unit profit. The product with the lowest unit profit requires a 
considerable amount of time on machine B and relatively little time on C. 
This cursory examination indicates that the maximum profit will not be 
achieved by restricting production to a single article. It would seem that 
at least two of them should be made. It is not too obvious, however, 
what the optimal product mix should be. 

Suppose x; is the number of units of product j produced per week. It 
is of interest to find the values of z1, тә, zs, 24 which maximize the total 
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TABLE 1-1 


DATA FOR EXAMPLE 


1 Products "Total time 
Machine type available per 
1 2 3 4 week 

A 1.5 1 2.4 1 2000 

B MB 1 3.5 8000 

С 1.5 3 3.5 1 5000 
Unit 
prone 5.24 7.30 8.34 4.18 


profit. Since the available machine time is limited, we cannot arbitrarily 
increase the output of any one product. Production must be allocated 
among products 1, 2, 3, 4 so that profits will be maximized without ex- 
ceeding the maximum number of machine hours available on any one 
of the groups of machines. б 

Let us first consider the restrictions imposed by the availability of 
machine time. Machines of type 4 are in use a total of 


1.521 Р £2 + 2.43 + x4 hours per week, 


since 1.5 hours are required for each unit of product 1, and x; units of 
product 1 are produced; and so on for the remaining products. Also, the 
total time used is the sum of the times required to produce each product. 
The total amount of time used cannot be greater than 2000 hours. Mathe- 


matically, this means that 
1.521 + aa + 2423 + T4 < 2000. (1-1) 


It would not be correct to set the total hours used equal to 2000 for 
type A machines, since there may not be any combination of production 
rates that would use each of the three groups of machines to capacity. 
We do not wish to predict which machines will be used to capacity. In- 
stead, we introduce a “less than or equal to” sign; the solution of the 
problem will indicate which machines will be used at full capacity. 

For machines B and C we can write 


zı + 5:9 + хз + 3.524 < 8000 (type B machines), (1-2) 
1.511 + 322 + 3.573 + z4 < 5000 (type C machines). (1-3) 


Since no more than the available machine time can be uséd, the variables 
zj must satisfy the above three inequalities. Furthermore, we cannot 
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produce negative quantities; that is, we have either a positive amount of 
any product or none at all Thus the additional restrictions 


Отот 0,07) ta 0 (1-4) 


require that the variables be non-negative. 
We have now determined all the restrictions on the variables. If 2; 
units of product j are produced, the weekly profit 2 is 


а = 524z, + 7.3025 + 8.3413 + 4.1814. (1-5) 


We wish to find values of the variables which will satisfy restrictions 
(1-1) through (1-4) and maximize the profit (1-5). 

The above example is clearly a programming problem. Moreover, it is 
a linear programming problem because the restrictions and the function 
to be maximized involve only linear relations among the variables. 

In practice, it may not be true that the profit derived from the sale of 
any one product is directly proportional to the number of units sold. More 
generally, the profit will be some function of the quantities produced, i.e., 


2 = f(x 22, ©з, 14). 


If this function is not of the form (1-5), we have a nonlinear rather than à 
linear programming problem. For example, if the profit function were 
of the form 

z = 524x1/? + 7.30210 48.3413? + 41571, (1-6) 


then the determination of the variables which satisfy the constraints (1-1) 
through (1—4) and maximize Eq. (1-6) would be a special case of a non- 
linear programming problem. 3 


1-4 Linear programming. The preceding example illustrated how a 
linear programming problem and a particular case of a nonlinear program- 
ming problem can arise in practice. Linear programming is concerned with 
solving a very special type of problem—one in which all relations among 
the variables are linear both in the constraints and the function to be 
optimized. The general linear programming problem can be described as 
follows: Given a set of m linear inequalities or equations in т variables, we 
wish to find non-negative values of these variables which will satisfy the con- 
straints and maximize or minimize some linear function of the variables. 

Mathematically, this statement means: We have m inequalities or 
equations in r variables (m can be greater than, less than, or equal to r) 
of the form: 


Qt, + tits +++ Fart, =, < Og t= 1,...,™ (1-7) 
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where for each constraint one and only one of the signs <, =, > holds, 
but the sign may vary from one constraint to another. We seek values of 
the variables x; satisfying (1-7) and 


z;20, Д (1-8) 
which maximize or minimize a linear function 
г = сүйү tes + 6g (1-9) 


The aij, bi, c; are assumed to be known constants. 

We have thus formulated the general linear programming problem 
which, in mathematical terms, can be represented by (1-7) through (1-9). 
A programming problem is linear if, in the constraints and function to be 
optimized, the variables appear only as linear forms. A linear form in- 
volving n variables xj is an expression of the type ал11 F -e + antn + b, 
where the a; and b are constants. It is very important to see what the 
assumption of linearity implies. Intuitively, linearity implies that products 
of the variables, such as Z172, powers of variables, such as 22, and com- 
binations of variables, such as a171 + аз log х2, cannot be allowed. 

In more general terms, linearity can be characterized by certain additive 
and multiplieative properties. In the context of the above example, 
additivity means: If we use tı hours on machine A to make product 1, 
and tz hours to make product 2, the time on machine A devoted to products 
1 and 2 ist, + ta. In this case, the additivity property seems quite reason- 
able if the time required to convert from product 1 to 2 is negligible. How- 
ever, not all physical processes behave in this way. If we mix several liquids 
of different chemical composition, it is, in general, not true that the total 
volume of the mixture is the sum of the volumes of the individual con- 
stituents. This is an example of a case where additivity may not hold. 

The multiplicative property requires: (1) If it takes one hour to make a 
single item on a given machine, it takes ten hours to make ten parts; this 
also seems quite reasonable. (2) The total profit from selling a given num- 
ber of units of a product is the unit profit times the number of units sold; 
this is not always true. In general, the profit is not directly proportional 
to the number of units sold even if the selling price is constant, since 
manufacturing costs per unit may vary with the number of units made. 
Thus the linearity implied in a linear programming problem is not always 
expected to be an absolutely accurate representation of the real world. 
Fortunately, the assumed linearity is often a close enough approximation 
of actual conditions so that it can provide very useful answers. 

One other important restriction is inherent in a linear programming 
problem: It is assumed that the variables x; can take on any values al- 
lowed by the restrictions (1-7) and (1-8); in other words, we cannot, for 
example, require that the variables assume only integral values. If the 
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additional restriction is imposed that the variables must be integers, then, 
in general, we do not have any longer a linear programming problem. 
Actual situations often require that the variables be integers; such problems 
are frequently solved by linear programming, and the answers are rounded 
off to the nearest integers which satisfy the constraints. This may or may 
not be a valid approximation. In general, the approximation is good if the 
solution requires that a large number of units of each variable be used. 
Since, in the example discussed in Section 1-3, the production is assumed 
to be continuous, it is not necessary that the optimal x; be integers. Of 
course, in practice no attempt would be made to schedule weekly produc- 
tion down to a fraction of a unit. 

The assumption that the variables can vary continuously goes somewhat 
deeper than indicated in the previous paragraph. Fundamentally, every- 
thing in the real world comes in discrete units, and nothing is infinitely 
divisible. However, the basic building blocks (molecules, photons, etc.) 
are often so small in comparison with the quantities under consideration 
that for all practical purposes (i.e., to ten or fifteen decimal places), it can 
be assumed that the physical quantity is continuously variable. The real 
difficulty appears when the discrete units are not small in comparison with 
the magnitudes of the variables. In situations of this kind, one must be 
concerned about the discreteness of the variables. 

The function to be optimized, (1-9), is called the objective function. Note 
that no constant term appears in the objective function, i.e., we do not 
write 2 = Ут, cjzj+ c. The reason for this is simple. The values of 
the x; which optimize г are completely independent of any additive con- 
stant c, Hence, if there is such a constant, it can be ignored during the 
process of determining the best ту, and added to г after the problem has 
been solved. 

Mathematically, the constraints (1-8) which require that the variables 
xj be non-negative do not differ from the constraints (1-7). However, 
when solving a linear programming problem, the non-negativity constraints 
are handled differently from the other constraints. For this reason, we 
shall refer to the non-negativity constraints as non-negalivity restrictions, 
while the term constraint will be used to denote constraints other than the 
non-negativity restrictions. Thus when we say that there are m constraints 
on the problem, we mean that there are m constraints of the form (1-7). 
Then in addition, there are the non-negativity restrictions. This termi- 
nology will save some confusion later, 

Any set of x; which satisfies the constraints (1-7) will be called a solution 
to the linear programming problem. Any solution which satisfies the non- 
negativity restrictions is called a feasible solution. Any feasible solution 
which optimizes the objective function is called an optimal feasible solution. 
The task of solving a linear programming problem consists in finding an 
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optimal feasible solution, Normally, there will be an infinite number of 
feasible solutions to a linear programming problem. Out of all these solu- 
tions, we must find one which optimizes the objective function. 


1-5 The transportation problem; another example. In practice, linear 
programming has been of particular significance in its application to so- 
ealled transportation problems. To provide the reader with a little more 
feeling for the nature of linear programming, we shall discuss these briefly. 
Later two chapters will be devoted to solving transportation problems. 

A typical transportation problem can be deseribed as follows: Given 
amounts of a uniform product are available at each of a number of different 
origins (e.g., warehouses). We wish to send specified amounts of the prod- 
uct to each of a number of different destinations (e.g., retail outlets), The 
cost of shipping one unit amount from any one origin to any one destina- 
tion is known. Assuming that it is possible to ship from any one warehouse 
to any one retail outlet, we are interested in determining the minimum- 
cost routing from the warehouses to the retail outlets, 

We shall suppose that there are m origins and n outlets. Take z;; to be 
the number of units shipped from origin ї to destination j. Note that here 
it is convenient to use a double subscript since it simplifies the notation. 
For a given i (warehouse), there are n possible j-values (retail outlets to 
which units can be shipped). Hence we have a total of mn different tij. 
Since negative amounts cannot be shipped, we must have z;; > 0 for all 
P 

Let a; be the number of units of the product available at origin ї and bj 
the number of units required at destination j. We cannot ship more goods 
from any one origin than are available at that origin. Hence summing 
over all destinations, we have 


n 
Уау = за tratue ttm Say, = 1,...,т. (1-10) 
jel 

‘There are m such constraints, one for each origin, We must supply each 

destination with the number of units desired; thus 


Dy ta med =}, 4-154 UM) 
iml 


The total amount received at any destination is the sum over the amounts 
received from each origin. The needs of the outlets can be satisfied if and 
only if 


SA a; > 5b. (1-12) 


rn j=l 


We assume that this is the case. 
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If суу is the cost of shipping one unit from origin 7 to destination j, then 
the total cost of shipment is 


n т 
ur 2 D ctu 


j=l i=l 
= (сихи eris ++ > + лаа) + (621221 tcc] + Conon) 
5-5 (nisi ho ++ + Cnnun): (1-13) 


The first term on the right of (1-13) is the cost of shipping from origin 1, 
the second the cost of shipping from origin 2, etc. We wish to find z;; > 0 
which satisfy the constraints (1-10), (1-11) and minimize (1-13). 

We can now summarize the transportation problem as follows: Find 


zij > 0 which minimize ae 
z= У) уусу» (1-14) 


j=l i=l 
subject to 
n 
Уу ы жш, с ves ich 
j=1 (1-15) 
m 
Уу) у= 5у d—deom 
ici 


This is a linear programming problem in mn variables with m + n con- 
straints. 

It will be observed that the constraints in the transportation problem 
have a particularly simple form. All the nonzero coefficients of the z;; are 
ones. Furthermore, any given z;; appears in two and only two of the 
constraints. Because of these special properties of the constraints, a trans- 
portation problem can be solved much more easily than a general linear 
programming problem оѓ equivaient size. Historically, this simplification 
made it possible to solve transportation problems before it was possible to 
solve general linear programming problems. At this point, it is also worth 
mentioning that many practical problems which have nothing to do with 
transportation can be formulated as transportation problems. 


1-6 Simple graphical examples. Linear programming problems which 
involve only two variables can be solved graphically. We shall illustrate 
the procedure by several examples, The geometrical interpretation of 
linear programming problems is very important, and we shall use it quite 
often in later work. An examination of the sorts of things which can occur 
in the simple cases involving only two variables provides a great deal of 
insight into what can happen in the more general case with any number 
of variables. 
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To begin let us find a geometric interpretation and solution for the fol- 
lowing linear programming problem:* 
3x, + 5r2 X 15, 
bz; + 222 < 10, 


21,22 > 0, 


(1-16) 


max г = 52; + 322. 


First, we shall find the sets of numbers (zi £2) which are feasible 
solutions to the problem. We introduce an z425-coordinate system and 
note that any set of numbers (zı, 22) is a point in the х12»-ріапе. All 
points (21, 22) lying on or to the right of the z;-axishavez, > 0. Similarly, 
all points lying on or above the z-axis have тә > 0. It follows that any 
‘point lying in the first quadrant has ту, zz > 0 and hence satisfies the 
non-negativity restrictions. Any point which is a feasible solution must 
lie in the first quadrant. 

To find the set of points in the first quadrant satisfying the constraints, 
we must interpret geometrically inequalities such as За; + 5х3 < 15. 
If the equal sign holds, that is 3r; + 522 = 15, we have the equation for 


ma 


Sn Bn + 5ш; = 15 
794 


Figure 1-1 


* To avoid monotonous repetition of the same group of words, we shall fre- 
quently represent linear programming problems in the abbreviated form (1-16); 
Eq. (1-16) should be read as follows: Find non-negative values of the variables 
zi, тә which satisfy the constraints 3zi + бхз < 15, 5x1 + 2x2 < 10, and 
which maximize the linear form 2 = 521 + 3z2. 
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a straight line, and any point on the straight line satisfies the equation. 
Now consider the point (0, 0), i.e., the origin. We observe that 3(0) + 
5(0) — 0 « 15, so that the origin also satisfies the inequality. In faet, 
any point lying on or below the line З=; + 5х2 = 15 satisfies 3r; + 
5z < 15. However, no point lying above the line satisfies the inequality. 
Therefore, the set of points satisfying the inequality Зх + 522 < 15 
consists of all the points in tbe z;za-plane lying on or below the line 
3z; + 512 = 15. The points which satisfy the non-negativity restrictions 
and this inequality are all the points in the first quadrant lying on or below 
the line 3z; + 5x3 = 15. In precisely the same way, we see that all 
points satisfying 5z; + 222 < 10 and the non-negativity restrictions are 
all the points in the first quadrant lying on or below the line 


bz; + 2х2 = 10. 


The set of points satisfying both Зх; + 5x? < 15 and 5z; + 2x2 < 10, 
as well as the non-negativity restrictions, is the set of points in the darkly 
shaded region of Fig. 1-1. Any point in this region is a feasible solution, 
and only the points in this region are feasible solutions. 

As yet we have said nothing about the objective function. To solve 
the linear programming problem, we must find the point or points in the 
region of feasible solutions which give the largest value of the objective 
function. Now for any fixed value of z, 2 = 5x; + 3x3 is a straight line. 
Any point on this line will give the same value of z. For each different 
'value of z, we obtain a different line. It is important to note that ali the 
lines corresponding to different values of z are parallel. This follows 
because the slope of any line 2 = сүгү + cox is —c1/cz and is independent 
of 2. In our problem с, cs are fixed and the lines are parallel. 

We should now be able to see how to solve the problem. We wish to 
find the line with the largest value of z which has at least one point in 
common with the region of feasible solutions. The parallel lines in Fig. 1-1 
represent the objective function for three different values of z. It is clear 
that 2; is not the maximum value of z because the line can be moved up, 
thus increasing z, while some of its points are still in the region of feasible 
solutions. On the other hand, although 23 > z2 and 21, the line corre- 
sponding to zg has no point in common with the region of feasible solutions; 
hence no feasible solution can yield as large a value of z. Thus we see that 
25 is the maximum value of z, and the feasible solution which yields this 
value of z is the corner A of the region of feasible solutions. 

The figure shows that the values of the variables for the optimal solution 
are approximately zı = 1, zı = 2.4. To find the exact values, we note 
that the point representing the optimal solution is the intersection of the 
lines 3z, + 5z5 = 15 and 52; + 2z, = 10. Solving these two equations 
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22 


21 29 23 


Figure 1-2 


simultaneously, we find that тү = 1.053, хә = 2.368. Substitution of 
these values into the objective function shows that the maximum value of 
zis 12.37. 

Now let us consider the problem: 


3r, + 5ra < 15, 


5a, + 2хә < 10, 
(1-17) 
21,23 2 0, 


maxz = 2.511 + 22. 


This problem is the same as the one just solved except for a slightly 
different objective function. The region of feasible solutions is the same as 
for the previous problem. Figure 1-2 shows the objective-function lines 
of (1-17) for three different values of z. 

° Clearly, 22 is the maximum value of z. Now, however, the line repre- 
senting the objective function lies along one edge of the polygon of feasible 
solutions. This means that there are no unique values of т, 22 which 
maximize z; any point on the edge AB of the polygon gives the optimal 
value of z. The maximum value of z is unique, but there are an infinite 
number of feasible solutions which yield this value of 2. Note that the 
corner which was optimal for the previous problem is also optimal for the 
present problem, Using these values of 21, 22, we find that max z = 5.0. 
For this problem, two corners, as well as any point on the line joining these 
two corners, are optimal solutions. When a linear programming problem 
has more than one optimal solution, we say that there are alternative 
optima; physically, this means that the resources can be combined in more 
than one way to maximize profit. 
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FicunE 1-3 


As an example of a problem for which the objective function is to be 
minimized consider: 
$i zx» < 4, 
бх; + 2x2 > 8, 
21 + öz > 


тү Š (1-18) 


4, 
3, 
22 X 3, 
21,12 > 0, 
minz = 2z, + 32. 


The geometrical interpretation of the problem is given in Fig. 1-3. 

The minimum value of z is гә. This minimum is found at а unique point, 
the point of intersection A of the lines бх, + 22: = 8andz, + 515 = 4. 
Solving these two equations simultaneously, we see that the optimal 
solution is z; = 8/7, тә = 4/7, and min z = 4. 

` Linear programming problems involving three variables can also be 
presented geometrically; however, their graphical solution is more difficult. 
As an illustration, we shall graphically solve the following problem in 


three variables: 
ату + 6x2 + 3r3 < 24, 


21+ 1.52: + 3x3 < 12, 
32 + z9 < 12, (1-19) 
21,49, 23 > 0, 


maxz = 0,57; + блр + бтз. 
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Ficure 1-4 


The region representing the feasible solutions is the shaded polyhedron 
in Fig. 1-4. We recall that an equation in three variables, such as 421 + 
бх» + 3rg = 24, is а plane. When the equality sign is replaced by an 
inequality (<), the set of points satisfying this inequality is the collection 
of all points lying either above or below the plane. The set of points 
(жү, хә, 23) which, when substituted into the objective function, yields a 
given value of z lies on the plane 2 = 0.521 + 6x2 + 513. As z is varied, 
a series of parallel planes is obtained. The plane representing the largest 
value of z which has at least one point in common which the region of 
feasible solutions gives тах 2. The point or points from the region of 
feasible solutions which lie on this plane are the optimal solutions. 

It is seen that the optimal solution occurs at corner A of the polyhedron 
of feasible solutions. At this point, the planes 4u, + бхз + 8x3 = 24 and 
a, + 1.512 + 323 = 12 intersect. However, zi = Oat A. Thus, by 
simultaneous solution of 622 + 3x3 = 24 and 1.572 + 825 = 12, тапа 
хз are found to be 22 = 7з = 22; the optimal solution is zı = 0, a2 = 
хз = 2%, and max z = 293. 


1-7 Some exceptional cases. The examples of the preceding section 
illustrate what may be called "properly: behaved” linear programming 
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zedr=6 z= 10 


Figure 1-5 


problems. Now we wish to show that there are certain exceptional cases 
which must be taken into consideration if a general technique for solving 
linear programming problems is to be developed. 

Let us study the following problem: 


тү — 232 


—05z, + z2 < 2, 


Iv 
I 


(1-20) 
în, 2 > 0, 


max z= 2r; + 223. 


The region of feasible solutions is the shaded area of Fig. 1-5. Lines 
representing the objective function for several values of z are also drawn. 
Clearly, we have encountered a new phenomenon. The line representing 
the objective function can be moved forever parallel to itself in the direc- 
tion of increasing z, and still have some points in the region of feasible 
solutions. Hence г can be made arbitrarily large, and the problem has no 
finite maximum value of z. In such a case we say that the problem has an 
unbounded solution. 

We do not expect any linear programming problem representing some 
praetical situation to have an unbounded solution, since this would 
imply, for example, the feasibility of an infinite profit. However, the 
limitation of resources and the impossibility of making arbitrarily large 
profits are precisely the reasons for our interest in using linear program- 
ming. Nonetheless, it occasionally happens that a mistake in the formula- 
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tion of an actual problem leads to an unbounded solution (the author 
has made such an error). \ 

In the above example, both variables сап be made arbitrarily large as 
zis increased. However, an unbounded solution does not necessarily imply 
that all the variables ean be made arbitrarily large as 2 approaches infinity. 
This can be seen by solving the problem: 

a $ 3, 
z س‎ < 0, 
21,13 2 0, 


(1-21) 


max z= —3r; + 222. 


This problem is presented graphically in Fig. 1-6. 

We have already noted that the set. of variables, which maximizes the 
objective function does not need to be unique. Now it may turn out that, 
although z has a finite maximum value, there are solutions giving this 
maximum z which have arbitrarily large values of the variables, ‘This is 
illustrated by the following example: 

xı 22 > ~1, 
206# + z2 < 2, 
21,23 > 0, 


max 2 = 213 — 71, 


(1-22) 
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FIGURE 1-7 


which is solved graphically in Fig. 1-7. This problem is not completely 
normal since there are solutions with arbitrarily large variables which 
yield the optimal value of z. The maximum value of z is 4. Furthermore, 
any point (zı, тә) lying on the edge of the region of feasible solutions, 
which extends to infinity, yields z = 4 and is therefore an optimal solution. 

Thus far, every example has had a feasible solution. However, there is 
no automatie guarantee that all linear. programming problems have 
feasible solutions. The next problem, shown graphically in Fig. 1-8, has 
no solution because the constraints are inconsistent: 


21 +22 X 1, 
2x; + 2:3 > 4, 


m (1-23) 
21,22 > 0, 


maxz = 3r; — 219. 


There is no point (x1, тә) which satisfies both constraints. 

The constraints can be consistent, and yet there may be no feasible 
solution because no point satisfying the constraints also satisfies the non- 
negativity restrictions. The following problem, illustrated geometrically 
in Fig. 1-9, gives an example of such a case: 


ay re > 0, 

Sz; — 25 € —3, 

RO eee Hd (1-24) 
21,29 > 0, 


maxz = zr, + 22. 
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———t = 1 


Figure 1-8 Ficure 1-9 


Any point in the shaded area satisfies the inequality constraints. However, 
since no solution has ту, хә > 0, there is no feasible solution. 

The reader should not feel that the examples presented in this section 
are of hypothetical interest only and never occur in the real world. While 
it is true that we do not expect any properly formulated practical problem 
to exhibit such behavior, it is equally true that a problem involving a large 
number of variables and constraints makes it extremely difficult to be sure 
that the constraints are indeed consistent, that there is a feasible solution, 
and that there is no unbounded solution. Because of this possibility of 
making mistakes in the formulation of problems, the exceptional cases 
discussed in this section can occur in practice. Hence the computational 
technique devised for solving a linear programming problem should be 
general enough to reveal the existence of an unbounded solution or the 
absence of a feasible solution. It would be most unfortunate if we had to 
ascertain that the problem actually possessed а finite maximum or mini- 
mum before we could proceed with the computation. 

There is one other exceptional case, i.e., the problem with a single solu- 
tion. However, it never presents any diffieulty in practice because the 
possibility of its existence is evident as soon as the problem has been 
formulated. It can oceur only if the number of equations in the constraints 
is at least equal to the number of variables. If the solution is feasible, it 
is optimal. If it is not feasible, the problem has no solution. A problem 
of this kind is of no interest as an optimization problem because there is no 
freedom; i.e., since there is only a single solution, there is nothing to be 
maximized or minimized. 


1-8 The simplex method. Let us summarize some of the properties of 
the linear programming problems which were solved graphically in the 
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preceding sections. Whenever feasible solutions existed, the region of 
feasible solutions bad what in geometry is called the convexity property. 
This geometrical term means that the regions had no holes in them, i.e., 
they were solid, and there were no indentations in the boundary. The 
fact that a region is convex can be expressed simply by saying that the 
line joining any two points in the region also lies in the region. Further- 
more, the boundaries of the regions were lines or planes. Finally, there 
were corners on the boundary, and there were edges joining the various 
corners. 

In addition, it was seen that for any fixed value of z, the objective func- 
tion could be represented by a line or a plane. We found the interesting 
result that whenever the maximum or minimum value of z was finite, at 
least one corner of the region of feasible solutions was an optimal solution. 
If the optimal solution was not unique, there were points other than 
corners that were optimal, but in any event at least one corner was optimal. 
The situation was different when the objective function could be made 
arbitrarily large. In this case, of course, no corner was optimal. As а 
matter of terminology unbounded solutions will not be called optimal. 
The term “optimal solution” will be taken to imply that the maximum or 
minimum value of z is finite. 

Interestingly enough, these observations, which were derived from 
simple graphical examples, hold true for the general linear programming 
problem if we think of it as being represented geometrically in an r-dimen- 
sional space. The region of feasible solutions is a convex region, or convex 
set. It also has corners, or extreme points, as they are called in the more 
general case. In addition, if there is an optimal solution, at least one of the 
extreme points will be optimal. 

The reader familiar with the methods of differential calculus and, in 
particular, with Lagrange multipliers, will probably feel that this tech- 
nique can be applied to solve any linear programming problem. Unfor- 
tunately, this is not true. The difficulty stems from the fact that the 
optimal solutions lie on the boundary of the region of feasible solutions 
and, worse yet, at corners of the boundary. It may be recalled that the 
methods of differential calculus determine relative maxima and minima; 
they are of considerably less value when absolute maxima and minima 
are to be obtained. A discussion of the reasons for this failure of the clas- 
sical methods would take us too far afield. In fact, if an optimal solu- 
tion exists, the differential calculus does tell us that one of the extreme 
points is optimal. Unfortunately, it does not specify which extreme point 
is optimal. 

Hence the idea of using the differential calculus to solve linear program- 
ming problems has to be discarded, and we must concentrate our efforts 
on developing a procedure which will exploit the special features of linear 
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programming problems. No method has been found which will yield an 
optimal solution to а linear programming problem in a single step. It is 
really not surprising that all the techniques for solving such problems are 
iterative: If there are numerous constraints, it can be extremely difficult to 
find any feasible solution, not to mention an optimal solution. (The 
reader familiar with numerical analysis may recall that some of the most 
efficient techniques for solving sets of simultaneous linear equations are 
iterative.) The best-known and most widely used procedure for solving 
linear programming problems is called the simplex method, which was 
developed by George Dantzig in 1947. The term “simplex” has nothing to 
do with the method as it is now used; it had its origin in a special problem 
that was studied in the early development of the method. 

The simplex method is an algebraic iterative procedure which will solve 
exactly (it is not an approximation method) any linear programming 
problem in a finite number of steps, or give an indication that there is an 
unbounded solution. The simplex method can be given a very simple 
geometrical interpretation in terms of the concepts that we have already 
introduced. We have stated above that if there is an optimal solution, 
one of the extreme points is optimal. There is only a finite number of 
extreme points. The simplex method is 4 procedure for moving step by 
step from a given extreme point to an optimal extreme point. At each step, 
it is possible to move only to what intuitively are adjacent extreme points. 
The simplex method moves along an "edge" of the region of feasible solu- 
tions from one extreme point to an adjacent one. Of all the adjacent 
extreme points, the one chosen is that which gives the greatest increase 
(or greatest decrease) in the objective function (in practice, for computa- 
tional reasons, one usually does not determine the extreme point which 
gives the greatest change in the objective function; instead one bases the 
selection on a simpler criterion which is almost as good). At each extreme 
point, the simplex method tells us whether that extreme point is optimal, 
and if not, what the next extreme point will be. If at any stage the simplex 
method comes to an extreme point which has an edge leading to infinity, 
and if the objective function can be increased (or decreased) by moving 
along that edge, the simplex method informs us that there is an unbounded 
solution. 

It was indicated above that the simplex method starts with a given 
extreme point. The question arises how to find an initial extreme point. 
In the examples of the previous two sections, it will be noted that the 
origin was often an extreme point of the region of feasible solution. If the 
origin is a feasible solution, it will be an extreme point because all variables 
are non-negative. Hence if z; — O = Agee sts satisfies the constraints, 
we have immediately an initial extreme-point solution. However, as the 
minimization problem of Section 1-6 has illustrated, it is not necessarily 
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true that the origin is an extreme point. In such a case, it is still possible 
to find an extreme point, but it involves considerably more work, as we 
shall see later. 


1-9 Brief historical sketch. Programming problems first arose in eco- 
nomics, where the optimal allocation of resources has long been of interest 
to economists. More specifically, however, programming problems seem 
to be a direct outgrowth of the work done by a number of individuals 
in the 1930's. One outstanding theoretical model developed then was 
Von Neumann's linear model of an expanding economy [7], * which was part 
of the efforts of a number of Austrian and German economists and mathe- 
maticians who were studying generalizations of Walrasian equilibrium 
models of an economy [8]. A more practical approach was made by 
Leontief, who developed input-output models of the economy [5]. His 
work was concerned with determining how much various industries would. 
have to produce to meet a specified bill of consumer demands. Input- 
output models did not actually involve any optimization; instead they 
required the solution of a system of simultaneous linear equations. 

During World War II, a group under the direction of Marshall K. Wood 
worked on allocation problems for the United States Air Force. Generaliza- 
tions of Leontief-type models were developed to allocate resources in such 
à way as to maximize or minimize some linear objective function. George 
B. Dantzig was a member of the Air Force group; he formulated the 
general linear programming problem and devised the simplex method of 
solution in 1947. His work was not generally available until 1951, when 
the Cowles Commission Monograph No. 13 was published [4]. 

After 1951, progress in the theoretical development and in practical 
applications of linear programming was rapid. Important theoretical 
contributions were made by David Gale, H. W. Kuhn, and A. W. Tucker, 
who had a major share in developing the theory of duality in linear pro- 
gramming. A. Charnes, who also did some important theoretical work, 
and W. W. Cooper took the lead in encouraging industrial applications 
of linear programming. 

Problems of the linear-programming type had been formulated and 
Solved before the pioneering work of Dantzig. In 1941 Hitchcock formu- 
lated and solved the transportation problem [1], which was independently 
solved by Koopmans in 1947 [3]. In 1942, Kantorovitch (Russian) also 
formulated the transportation problem [2], but did not solve it. The 
economist Stigler worked out a minimum-cost diet in 1945 [6]. Although 
this problem can be formulated as a linear programming problem, Stigler 
did not use this technique. It was not until Dantzig's work, however, 


* Numbers in brackets refer to bibliographical references. 
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that the general linear programming problem was formulated as such, 
and a method devised for solving it. " 


1-10 The road ahead. ‘The material in this book assumes that the 
readeg has some familiarity with the theory of vector spaces, matrices, and 
convex sets. Chapter 2 summarizes the material needed. No attempt is 
made, however, to provide a rigorous or complete development. The 
reader who has had no previous acquaintance with linear algebra may find 
it desirable to study simultaneously a more complete development of the 
material. The author’s companion volume, entitled Linear Algebra, was 
written for this purpose and is suggested as collateral reading. 

Chapter 3 begins the formal development of linear programming and 
presents the fundamental results needed in the derivation of the simplex 
method. Chapters 4 and 5 consider the computational details of the sim- 
plex method and the geometrical interpretation in the so-called solutions 
and requirements space. The problem of degeneracy is studied in Chapter 
6, where the perturbation technique and the generalized simplex method 
for resolving the theoretical problems associated with degeneracy are dis- 
cussed, The revised simplex method is presented in Chapter 7. Chapter 8 
considers duality and presents two additional techniques for solving linear 
programming, problems, the dual simplex method and the primal-dual 
algorithm. The properties of transportation problems are investigated in 
‘Chapter 9. These properties allow the development of two special pro- 
cedures for solving transportation problems, which are also presented in 
this chapter. Chapter 10 discusses network-flow problems and develops 
the efficient network-flow techniques for solving transportation problems. 
A number of miscellaneous topics, such as parametric programming and 
upper bounds, are covered in Chapter 11. Chapters 12 and 13 discuss 
applications of linear programming. Applications to industrial problems 
are considered in Chapter 12, and applications to economies in Chapter 13. 
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PROBLEMS 


Solve the linear programming problems 1-1 through 1-12 graphically and 
shade in the region representing the feasible solutions. Hint: By testing one. 
point not on the line, it is always easy to determine which side of a line aizi + 
222 = b corresponds to aizi + азго > b (ог < b). The origin is often a good _ 
point to test. 


1-1. 2x; + 322 € 6, 1-2. bz; + 10r2 < 50, 
zı + 4x2 X 4, 114222 1, 
zı z2 > 0, za S 4, 
maxz = zi + 1.5r2. zi, 2 > 0, 
maxz = х] -} хә, 
1-8. бху + 10хә < 50, 1-4. zim z2 > 0, 
zi-- 22 > 1, 0:521 + 22 < 1, 
2 € 4, z1,22 > 0, 
2ı, r2 > 0, maxz = ту — 0.752) 
minz = 2r; + z2. 
1-5. zi + 312 > 3, 1-6. = +2221, 
т +222 2, тз < 5, 
1222 0, zi S 6, 
minz = 1.521 + 2.52. 7x1 + 92 < 63, 
zj,22 > 0, 
maxz = 10z; + 6.250. 
1-7, 211 + 22> 1, 1-8. zı r2 0, 
321 + 422 > 1.5, zı ¬ 519 > —5, 
71,22 > 0, zı, #2 > 0, 
minz = 62; + 479. minz = 2r, — 10r2 
1-9, zi or < 2, 1-10. zı — 0.522 > 0, 
=z; — 5r > —10, zı — 5т2 > —5, 
21722 > 0, zi, r2 > 0, 
maxz = —bzs. minz = т — 10r». 
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1-1. 422 <1, 1-12, nz: > 1 
—0.5r, — 522 < —10, з — br S 0, 
zı z2 > 0, bn — 12 0, 

maxz = — А22. z—2z222 —1, 
" zid zs, 
zi 8, 
11,22 > 0, 


1-13. Examine carefully the following pairs of problems: (a) 1-1 and 1-7; 
(b) 1-8 and 1-9; (e) 1-10 and 1-11. How are the solutions related? Each pair 
of problems forms what is called a set of dual problems, Note that both dual 
problems involve the same constants, but in rearranged order. 


CHAPTER 2 
MATHEMATICAL BACKGROUND 


"What's the good of Mercator’s 
North Poles and Equators, 
Tropics, Zones and Meridian Lines?” 
So the Bellman would сту: 
And the crew would reply 
“They are merely conventional signs!” 


Lewis Carroll, The Hunting of the Snark, Fit IL. | 


2-1 Matrices. Our later discussions may be considerably simplified 
by the use of some elementary topics from the theory of linear algebra. 
Here we shall briefly present the material from linear algebra which will 
be needed. No attempt at a rigorous development will be made.* 


Marrix: A matrix is defined to be a rectangular array of numbers and 


is written as follows: 
аі 012*** Ain 


азі G22°** Gan 
А = |а| = |; гө (2-1) 


Ami Om2*** mn 


The above array is called an m by n-matrix (written m X n). Brackets [ ], 
parentheses (), or the form || || is used to enclose the rectangular array 
of numbers. A matrix has no numerical value; it is simply a convenient 
way of representing arrays (tables) of numbers. The numbers in the 
array are called the elements of the matrix. A double subscript is used 
to denote the location of any given element. The first subscript gives the 
row and the second subscript the column in which the element is located. — 
Thus a;; is the element in row 7 and column j. 

Mairices will usually be denoted by upper-case boldface roman letters 


(A, B, etc.), and elements by italicized lower-case letters (aij, bij, etc.), i 


unless specific numbers are used. Note that in (2-1), the expressions - 
A and |la;jl| do not indicate how many rows or columns the matrix has. 
When only the typical element a;; of the matrix is shown, we use |[a;;|| 
rather than (a;;) or [a;;]. 


* The background material needed is developed in detail in the author's 
text Linear Algebra. References to sections in that work will be. given in ab- 
breviated form; for example, (LA 6-3] means Section 3 of Chapter 6. 
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We shall adopt the convention that brackets will be used to enclose 
matrices having at least two rows and two columns. Parentheses will 
be used to enclose a matrix of a single row. To simplify the printing in 
the text of matrices having a single column, they will be printed as a 
row and will be enclosed by brackets. In equations and in examples a 
matrix of a single column will sometimes be printed as a column for 
added clarity. 

Any matrix which has the same number of rows as columns is called 
a square matrix. A square matrix with n rows and n columns is called 
an nth-order matrix. 

Two matrices A, B are equal, written A = B, if and only if the correspond- 
ing elements are equal, that is, aij = bij for every i, j. The sum C of a matrix 
A and a matriz B, written C = А + В, is a matriz whose elements are given 
by су = aij-- bi. Two matrices A, B cannot be added unless the 
number of rows in A is equal to the number of rows in B and the num- 
ber of columns in A is equal to the number of columns in B. It is obvious 
that matrix addition satisfies the commutative and associative laws, 
that is, 

A+B=B+A, (2-2) 


А+(В+С) = (А+В)+С=А+В+С (2-3) 


since addition proceeds by elements and the laws hold for real numbers. 
To multiply a matriz A by a real number №, written AA, each element of A 
is multiplied by №. Thus АА = |[Na;j]|. Note that AA = Ad. A real num- 
ber will often be referred to as a scalar, and hence in forming XA, we say 
that A has been multiplied by the scalar A. 


EXAMPLES: 
( А [2.3.5], bii 
120 0 2 1 
Аъв- [% 10.11]. 
Mui eet 
54 2549 
(0) A-|s 9], 3205; = |15 45|-- 
8 1 4 05 


The product AB of two matrices A, B is defined if and only if the number 
of columns in A is equal to the number of rows in B. If Aisanm X r matriz 
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and B is an r X n matriz, then the product С = АВ is defined, and С is 
an m X n matrix whose elements are computed from 


3 
(uL RUE. E lesa gue od e. (2-4) 
k=l 


EXAMPLE: 
2 4 
Az|3.2]; B= |, А х 
4g Өн 
ер us 2(3) + 1(0) 2(2) + 1(1) 6775 
AB — 8.2 k : = |3() +20) 3(2) +21) | = 3 8 
4 5 i 4(3) + 5(0) 4(2) + 5(1) 12-18 


Note that BA is not defined. 
Matrix multiplication satisfies the associative and distributive laws: 
(AB)C = A(BC) = ABC (associative law), (2-5) 
A(B + C) = AB + AC (distributive law) (2-6) 


when the appropriate operations are defined. However, in general, matrix 
multiplication is not commutative, ie., in general, AB and BA are not 
the same thing. The above example shows that AB and BA are not 
necessarily both defined. However, even when AB and BA are defined, 
they need not be identical. 


2-2 Special matrices. In this section we wish to present a number of 
important definitions. 


IDENTITY MATRIX: The identity matrix of order т, written Y or In, is 
a square matrix having ones along the main diagonal (the diagonal running 
from upper left to lower right) and zeros elsewhere: 


1 0 0...0 
0 1 0.-.0 
1= |o 0 1---0}- (2-7) 
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If we write I = [18:31|, then 


Mmi" t7 (2-8) 
0 tj. 
The symbol à;; defined by (2-8) is called the Kronecker delta. The 
symbol will always refer to the Kronecker delta unless otherwise specified. 
It is tacitly assumed that in (2-8) the indices ї, j run from 1 to n. Fre- 
quently, we must, deal with several identity matrices of different sizes. 
Differentiation of such matrices is facilitated by writing I,, where the 
subscript indicates the size of I. 
Direct multiplication shows’ that if A is an m X n matrix, then 


TA s AI, = А: (2-9) 
If we write I4L, = 2, then I? = I or, in general, I* = I, k = 1, 2,.... 


For any scalar A, the square matrix S = |[A4;]] = А is called a scalar 
matrix. The square matrix D = ||A;à;;]| is called а diagonal matrix. 


Nut MATRIX: A matrix whose elements are all zero is called a null or 
zero matriz and is denoted by 0. 


A null matrix does not need to be square. When the operations are 
defined; we have 

A+0=A=0+A; A—A=0; AO=0; ОА = 0. (2-10) 
If a, b are real numbers, then ab = 0 implies that a = 0, or b = 0, or a 


and b = 0. But the matrix equation AB = 0 does not imply that A = 0 
or B = 0. It is easy to find non-null matrices whose product AB = 0. 


EXAMPLE: l4 4 о] [o o]. 
0 011—1 0 0 0 


Transpose: The transpose of a matriz A = |la;gl| is a matrix formed 
from A by interchanging rows and columns such that row i of A becomes 
column i of the transposed matrix. The transpose is denoted by A’; if 
Al = |aijll, aij = ау. 


From the definition of the transpose it is easy to prove the following 


relations: 
(A +B)’ = A’ +B’, (2-11) 
(AB)' = B'A', (2-12) 
Y=], (2-13) 


(A^) = А. (2-14) 


28 MATHEMATICAL BACKGROUND [cuar. 2 


EXAMPLES: 
10 
а) maf А jJ then A^ = |з 1]. 
010 4 0 


(2) wasp ae б, jJ 
0 5 1*2 


then ap =|? M ase [^ | 


NAE И; Tg e ied HAAS 17 VEA [ы s| = emn 
3.5 42 10 10 


SYMMETRIC MATRIX: A symmetric matrix is a matriz А for which A! = A. 


A symmetric matrix must be square, and aj; = aj; for all î, j. A skew- 
symmetric matrix is a matrix A for which A = —A’. A skew-symmetric 
matrix is also square and aj; = —aj; all i, j. Hence the diagonal ele- 
ments of a skew-symmetric matrix are zero, 


2-3 Partitioning of matrices. We begin this section by defining a 
submatrix, 


Supmarrrx; If we cross out all but k rows and s columns of an m x n 
matrix A, the resulting k X s matrix is called a submatrix of A. 


ExawPLE: If we cross out rows 1, 3 and columns 2, 6 of the 4 x 6 
matrix A = [[а;, we are left with the 2 x 4 submatrix 


pe: M23 аң fee 
d41 Ояз аз, 045 


For many reasons, a number of which will become clear later, it is at 
times useful to partition a given matrix into submatrices. Consider the 
matrix 


Амын oa (2-15) 
431 032 | 033 
441. 042 | а 
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By drawing the dotted lines shown we have partitioned A into four sub- 
matrices which are: 


a [tS] ае [er en]: 


аз 022 аза 042 
к 8р лае [Әр eom 
Then А can be written: 
A= pn es (2-17) 
An Аз; 


When A is written in the form (2-17), we say that A has been written as 
a partitioned matrix. Clearly there are many different ways in which A 
could be partitioned. 

Frequently, partitioning is used to simplify operations with matrices, 
such as multiplication. If partitioning is to be of advantage, we would 
like to be able to multiply by “blocks” and have the rule for multiplication 
follow the usual rule, Thus, if AB is defined and A and B are parti- 
tioned as 


Ay А; 
Aw وا‎ |i mid t (2-18) 
To Ba, Bo 


we wish to be able to write AB as 


АВ + АВ; АВ; + А2822 
AB = |А;В + А,В. А.В + А›;В;;/- (2-19) 
Аз\В + АзәВ AgiBia + Аз;В;; 


This will be correct if each product AaB, ія defined, In order that each 
such product be defined, the columns of A must be partitioned in "the 
same way” as the rows of B. 

A particularly useful form of partitioning is that in which each sub- 
matrix is а column of the given matrix (or а row of the given matrix). 
We shall find it convenient to denote matrices containing a single row 
(row matrices) or column (column matrices) by lower-case boldface 
letters rather than upper-case boldface letters. Each column of an m X n 
matrix A can be considered to be a submatrix of one column and m rows. 
The jth column of A will be denoted by aj Thus а, = [aij . .. ‚а^. 
Similarly, each row of A сап be considered to be a submatrix of one 
row and n columns. The ith row of A will be denoted by a‘ so that 
a = (041,:::; ам). By partitioning into rows or columns we can write 
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a matrix A as a row of column matrices, i.e., A = (а1,...,а,), or asa 
column of row matrices, ie, А = [a!,...,a"] _ 

Consider the matrix C = AB where C is m X n, A is m X r, and B 
isr X n. Then еу = X: aibi; or, if c; is the jth column of C, we have 


cj = [= aufi, Y: andis У =] = Ab; (2-20) 
k k k 


where b; is the jth column of B. Thus, if we write B = (b,,..., bn), 
then C = (Ab;,..., Ab,). Each column of C is computed by multi- 
plying the corresponding column of B by A. 


2-4 Determinants. Associated with every square matrix A is a number 
which is called the determinant of A, Only square matrices have determi- 
nants associated with them. Here we shall briefly summarize some useful 
properties of determinants without making any attempt to provide proofs. 
The complete development is found in {LA 3-10 through 3-17]. 

A set of integers 1, ...,n are in “natural order" when they appear in 
the order 1, 2, 3,...,7. If two integers are out of natural order in а 
set of m integers, then a larger integer will precede a smaller one. For 
example, the natural order of the first five integers, beginning with 1, 
is (1, 2, 3, 4, 5). When the integers 2 and 4 are interchanged, we obtain 
(1,4, 3, 2, 5). The set is now out of natural order because 4 precedes 3, 2, 
and 3 precedes 2. Any rearrangement of the natural order of n integers 
is called a permutation of these-integers. The interchange of two integers, 
such as 2 and 4 in the above example, is called a transposition. The num- 
ber of inversions in a permutation of n integers is the number of pairs of 
elements (not necessarily adjacent) in which a larger integer precedes a 
smaller one. In our example, there are three inversions: (4, 3), (4, 2), 
and (3,2). It should be noted that the number of inversions in any 
permutation is unique, and can be counted directly and systematically. 
A permutation is even when the number of inversions is even, and odd 
when the number of inversions is odd. 


DETERMINANT: The determinant of an nth-order matriz A = |lai;||, 
written. |A], is defined to be the number computed from the following sum 
involving the n? elements in A: 


[А| = X:(3:)21:25; . . ал, (2-21) 
the sum being taken over all permutations of the second subscripts. A 


term is assigned a plus sign if (i,j,...,7r) is am even permutation of 
(1,2,...,n), and a minus sign if it is an odd permutation. 
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Each term in (2-21) is the product of n elements from A, one from each 
row and column. There are n! such terms in (2-21). We shall find it 
convenient to refer to the determinant of an nth-order matrix as an 
nth-order, or n X n, determinant. Often we shall write |4| as 


ау... O1n 


ЈА = |? , (2-22) 


алі... пп 
with straight lines denoting the determinant (instead of brackets). ` 


ExaMPLE: From (2-21) the determinant of a third-order matrix A = 
lai] is 


|А| = 122033 — 412421033 + 012023031 — 013022031 
+ a13ā21032 — 41102303». (2-23) 


The term 44,2243, in which the second subscripts are in their natural 
order, represents the identity permutation of the second subscripts, i.e., 
there are no inversions. The identity permutation is even. 
It can be shown that the determinant of an nth-order matrix A can 
be written 
a 
IA] = 220545 (2-24) 
j=l 
for every i = 1,...,n, where Aj; is (—1)'*4 times the determinant of 
the submatrix obtained from A by crossing off row ¢ and column j; Aij is 
called the cofactor of element ахз. The expression (2-24) is called the cofactor 
expansion of |A| by row i. Note that for a matrix of a single element, 
А = (ал), Eq. (2-21) implies that |A| = а. ‘Thus if (2-24) is to hold 
for n = 1, we must define the cofactor of the element in a 1 X 1 matrix 
to be 1. It is also possible to expand in cofactors by columns. Thus 


ЈА = 22 aids, (2-25) 
inl 
for every j = 1,...,n. 
ExAMPLES: (1) Expanding by row 2, we see that, 


апу 412 413 
аз G22 423 
азі 432 433 


[| = = аз1А21 + 022422 + 023423, (2-26) 
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3/412 G13 
and An = (—1) | = (—1)(a12433 — 13035), 
a32 433 
4 |011 013 
As» = (—1) | | = 411033 — 413431, 
азі 433 
5/411 G12 
Азз = (—1) | | = —(@11032 — 212031). 
азі аз? 


Substitution of the 4;; into (2-26) gives (2-23). We note that the co- 
factor expansion or (2-21) shows that 


bir bie 
bar boo 


which is the expression learned in elementary algebra for a 2nd-order 
determinant. 


[В| = 


= Буба — bibo, 


(2) Expanding in cofactors by the first row, we see that 


IL] = (70"lL i]. 
Thus 
l=, 1. 


The expansion by cofactors reduces the problem of evaluating an 
nth-order determinant to that of evaluating n determinants of order 
т — 1, ie., the cofactors A;;. Thus proceeding by steps, we arrive at an 
evaluation of |A|. Application of the cofactor-expansion method to the 
cofactors reduces the task of evaluating each cofactor of order n — 1 to 
that of evaluating n — 1 determinants of order n — 2, ete. The expansion 
by cofactors is not a very efficient numerical procedure for evaluating 
determinants of high order. It is, however, of considerable theoretical 
value. 

We find it useful to give a name to the kth-order determinants, 
к = l,...,n — 1, which appear in the step-by-step evaluation of JA] 
by cofactor expansion, They will be referred to as minors of A. More 
precisely: 


Міхов or ORDER k: For апу m X n matriz A consider the kth-order 
submatrix R obtained by deleting all but some k rows and k columns of A. 
Then |R| is called a kth-order minor of A. 


Note that in the above definition A is not required to be a square matrix. 
We shall now summarize some useful properties of determinants: 


(1) An interchange of two columns in an nth-order matrix A changes 
the sign of |A]. 

(2) An interchange of two rows in an nth-order matrix A changes the 
sign of |4|. 


1 
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(3) |A’| = JA]. 

(4) Multiplication of each element in the ith row (or ith column) of 
an nth-order matrix A by a scalar à multiplies [А| by А. Thus |AA| = 
X'|A|, and hence |-A| = (—1)"|A]. 

(5) If every element in the ith row or ith column of A is zero, then 
|A| = 0. 

(6) Addition of a multiple of row k to row ? (i = k) or of a multiple 
of column k to column i (i з k) of A does not change the value of |A]. 

(7) The determinant |A| vanishes if A has two rows or two columns 
which are the same, that is, a; = ау (all К) or а = aij (all 7). 

(8) If A, B are nth-order matrices, then |AB| = |A| |B], i.e., the de- 
terminant of the product is the product of the determinants. Li 


Property (7) can be used to obtain an important result. Consider 
У; aijAnj. We are using the cofactors of row k and the elements of row 7. 
This is exactly the cofactor expansion of the determinant of a matrix 
whose rows č and k are the same, Hence 


D Ak; = MasAn z0 (i = k). (2-27) 
j j 


Combining (2-27) with (2-24) and (2-25), we obtain 
Уу) ауАь = У) анйљ = |A| des (2-28) 
Я j 
where 5j; is the Kronecker delta defined earlier. 


It will be noted that (2-28) looks very much like a matrix product. 
If we set aj; = Ау, then (2-28) becomes 


D) aya. = » ajjaj; = |A| би. (2-29) 
i j 


Thus if 
An Aa: Ant 


A sau à ^^ (2-30) 


Ain Aan’ Ann 
then Eq. (2-29) can be written in matrix form: 
AA+ = АТА = |AlL.. (2-31) 


The matrix At is called the adjoint of matrix A. In fact, A* is the trans- 
pose of the matrix obtained from A by replacing each element a;; by its 
cofactor Ajj. 
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2-5 The matrix inverse. Another important concept in matrix theory 
is the matrix inverse. We define: 


MATRIX INVERSE: Given a square matrix A. If there exists a square 
matrix B which satisfies the relation 


AB = BA =I, (2-32) 
then B is called the inverse of A. 


Usually, the inverse of A is written A7". Only square matrices have 
inverses. 

We essentially obtained an explicit expression for the inverse of A in 
(2-31). If we write 


1 
AT => at, 2-33 


then from (2-31) we see that A7" satisfies (2-32) and hence is the inverse 
of A. Note that A^! in (2-33) exists only when |A| = 0. 


SINGULAR AND NONSINGULAR MATRICES: The square matrix A is said 
lo be singular if |A| = 0; nonsingular if |A| з 0. 


We have shown above that every nonsingular matrix has an inverse. 
It is also true that only nonsingular matrices have inverses. 


ExawPLEs: (1) Let us compute the general formula for the inverse of 
a 2 X 2 nonsingular matrix A: 


ац orl A 
= 1 = 041022 — 012421. 
Ln azal IAl 


Then, since the cofactors are determinants of order one with values 
Аџ = 425, Aig = —4251, Азр = —ацо, As2 = ayy, it follows that 


е У atleast ul 


КОЕ]. 
2 3 
Then 


|A| = 3, к | 3 |J and =| l zi 
-2 1] IS 


(8) I= theta 


(2) Assume that 
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(4) If A = (ау), then |A| = an, А? = (1), and A^! = (1/211) 
а # 0. 


When a matrix is a single element, the inverse is also a single element, 
i.e., the reciprocal of that in A. There is a complete correspondence be- 
tween matrices of a single element and numbers. We shall not distinguish 
between the matrix A = (а) and the number «ıı. 

The following is a list of useful properties of the inverse: 


(1) The inverse of a nonsingular matrix is unique, ie. there is only 
one matrix for which (2-32) holds. To prove this suppose that there 
existed two different inverses, B and D, for A. By assumption, AB = I. 
Hence DAB = DI = D. However, by assumption, DA = I. Hence 
В = р. 

(2) If we have two nth-order matrices A and B such that AB = I, 
then A and B are nonsingular, A7! = В, B^! = A, and BA = I. To prove 
this we note from property (8) of determinants that, |A] [В| = Ш = 1. 
Hence both A and В are nonsingular. Thus А! exists. Multiplying 
АВ = Ion the left by A7", we obtain B = A7! since A~'A = I. Conse- 
quently, from (2-32), BA — I. 

(3) If A, B are two nth-order nonsingular matrices, then (AB)! 
B-!A-!, This is proved by noting that ABBA"! = АТАС! 
AA”! =I. By properties (1) and (2), the result follows. 

(4) (A7)7! = A, This follows immediately from (2-32) because if 
B is the inverse of A, A is also the inverse of B. 

(5) (A=) = (A )'. 

(6) If A is nonsingular and AB = 0, then B = 0, To prove this multi- 
ply on the left by A™™. This gives АС!АВ = A70 or B = 0. 


ug 


2-6 Computation of the inverse by partitioning. Suppose that we 
have an nth-order nonsingular matrix M which is partitioned as follows: 


M= [ и; (2-34) 
Y 6 


where « is an s X s submatrix, 8 an 8 X m submatrix, Y an m X 8 sub- 
matrix, and 6 an m X m submatrix (n = т + з). We use here Greek 
letters and no subscripts for the submatrices in order to simplify the 
formulas to follow. МГ} exists and will be partitioned in the same way 


as M, that is, 
м- = Е a (2-35) 
ср 


where A is s X з, B is s X m, C is m X s, and D is m X m. 
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Assume that 6 has an inverse, and that 6^! is known. Then, since 


MMT = I, ' 
edo um 
Y 8][C р 0 L, 


Four equations are obtained for the four unknown submatrices A, B, 
C, D: 
аА 8C = 1,  oB- D = 0, 


(2-37) 
ТА + 6С = 0, УВ + 0 = In. 
Solving these for A, B, C, D, we obtain 
22 — 8- y)! MMC. —1 
A= (a — B6^!v)^!, Apo, (2-38) 


C= SIYA м D = 65! — 6 YB. 


Since М^'! exists, the submatrices A, B, C, D exist. Hence, if 87! exists, 
all the operations can be carried out, and A, B, C, D can be computed 
from (2-38). 

Imagine now that we wish to invert a matrix by the partitioning scheme 
introduced above. If we partition the matrix so that ё is of a size which 
can be inverted easily, then we shall find it difficult to obtain A if the 
order of the given matrix is fairly large. We are able to avoid this dif- 
ficulty by applying the partitioning technique two or more times. Initially 
we partition the given matrix so that A is easy to compute. To find 67! 
we apply the partitioning technique one or more times to 6. 

A special case that will be of interest to us arises when M in (2-34) has 


the form 
E (2-39) 
oR 
where R~! exists and is known. Then from (2-38) we see that 
Ж -1 
urere]. (2-40) 
com 


2-7 Power series expansion of (I — A)T}. For the square matrix A, 
we shall write AA = A?, AAA = A, ete. It is also convenient to write 
A" = І, Now note that 


(I — ALF AHA т... 45 — p дЕ (2-41) 
By lim АК = 9 (2-42) 


kee 
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we shall mean that each element of A**! approaches zero as k becomes 
arbitrarily large. If (2-42) holds, then (2-11) becomes 


(1—A) 3; A 21 "or (Aye I+ A+ AP + +. (2-43) 
k=0 
Thus if (2-42) holds, (I — А)! exists, and can be written as a matrix 
power series. 
We shall now present an important set of conditions which guarantee 
that (2-42) holds. If for the matrix A, 


0 < aij <1 foralli,j and due, j*9 1,:.-,9,. (2-44) 
Шет 

then (2-42) holds. To prove this, let a{j be the ith element of AF. Then 

by (2-44) there exists an r (0 < r < 1) such that Sy aij € r for all j. 

Hence if A? = (14|, then 


* ^ "P 

a? = ana; Уа = У) Уа sr’, all 5j, (2-45) 
kewl iml iml kewl 

so that a? < r. Continuing this process, we see that aff < r. Conse- 

quently, each af? approaches zero as k — ж. 


2-8 Vectors and euclidean spaces. Matrices having a single row or 
column are often referred to as vectors. A matrix of a single row is called 
a row vector, and a matrix of a single column is called a column vector. 
Vectors will usually be denoted by lower-case boldface type. A vector а = 
(@ı, ..., an) ога = [а1,... 4] is called an n-component vector, and 
aj i = 1,...,N, is said to be the ith component of a. 

Vectors can be given a geometric interpretation. Just as (a), аз, аз) 
can be considered to be a point in a three-dimensional space, so can 
a = (a,,...,a,) be imagined to be a point in an n-dimensional space. 
There is no geometric distinction between row and column vectors. 
Geometrically, they are considered to be equivalent, The decision of 
writing a vector as a row or as a column is merely a matter of notational 
convenience. We shall more frequently use column vectors than row vec- 
tors, and hence a vector will, in general, be written a = [ar .. +, an), 

The operations of addition and multiplication by a scalar for vectors 
can be given a simple geometric interpretation in two and three dimen- 
sions, To illustrate these relations most clearly it is helpful to represent 
a vector by a directed line drawn from the origin to the point which 
characterizes the vector. An arrowhead is placed at the end point of the 
line, We shall often find it helpful to represent vectors geometrically by 
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T3 


(2ay, 2ag) 


s (ay, a2) 


TE 


FIGURE 2-1 


z2 


4 
اا م ی‎ iat ы-„ (a; + by, aa + ba) 


Ti 


directed line segments. The operation of multiplication by a scalar is 
illustrated geometrically in Fig. 2-1, while addition is illustrated geomet- 
rically in Fig. 2-2. Note that if c = a + b, then c is the diagonal of the 
parallelogram with sides a, b. 

Certain vectors which are used frequently are given names, and hence 
we shall define them at the outset. 


Uxır vecror: A unit vector, denoted by е;, is a vector with unity as the 
value of its ith component and with all other components zero. 


For vectors with n components there are n unit vectors. They are: 
€i = [1,0,...,0], e; = 10,1,0,...,0]..., 6, = [0,0,..., 1]. 
(2-46) 


NULL vector: The null vector, or zero vector, written 0, is a vector all of 
whose components are zero. 
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Sum VECTOR: А sum vector is а vector having unity as a value for each 
component; it will be written 1. f 


Ys (11, , .., 1). (2-47) 


The sum vector will be used more often as a row vector than as a column 
vector. The matrix product of 1 and a column vector a = [a,..,, an] 
yields 1а = У; а. This explains why 1 is called the sum vector. 


We shall occasionally need vector inequalities. Given two n-component 
vectors a and b, then a > b means aj > bi, 1 = 1,...,n, and a <b 
means a; € b; = 1,...,7. Similarly, a > b means a; > b; for all i, 
and a < b means а; < b; for all s. 

The scalar product of two n-component vectors a, b is defined to be 
the number 


Dd abi (2-48) 


tml 


This is true, irrespective of whether a and b are row or column vectors. 
If a isa row vector and b is a column vector, the scalar product is de- 
noted by ab; if a, b are column vectors, the scalar product is denoted 
by a'b; if a, b are row vectors, the scalar product is denoted by ab’, In 
every case the notation for the scalar product is such that if it is inter- 
preted as a matrix product, the result is (2-48). 

We have observed that a = [ay,..., ал) can be interpreted as a point 
in an n-dimensional space. As yet we have no coordinate system for this 
space. Now note that a can be written in terms of the unit vectors as 


follows: 
а = ауе, + aseo + ii + ases. (2-49) 


In two- and three-dimensional spaces, e, is а point located a unit distance 
from the origin on the ith coordinate axis. Thus for our n-dimensional 
space, we use the unit vectors to define a coordinate system. We imagine 
that a coordinate axis is drawn from the origin through each unit vector. 
The origin is the null vector. For this coordinate system, а; is the ith 
coordinate of the point a. Thus, when we write а = [a1, . . . , а], а special 
coordinate system is implied, i.e., the coordinate system defined by the 
unit vectors. 

The distance between two points а = [а1, . . . , an] and b = [bi, . .. , bal, 
written |a — bj, is defined to be 


ja — b| = [a — b(a — 5i = È d- |". (2-50) 
i=l 
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This is a direct generalization of the formula for distance in three dimen- 
sions. The distance from 0 to a is called the length or magnitude of a, 
and is symbolized by |a. The angle 0 between two nonzero vectors is 
computed from 


аъ и 
cos 6 = fal ibi . (2—51) 


It can be shown that |cos 0| € 1, so that it is always possible to de- 
termine an angle 0 lying between 0 and т. To show that |cos 0| < 1 is 
equivalent to showing that 


la'b| < |а| || (2-52) 


for any two n-component vectors a, b, where |a’b| is the absolute value’ 
of a'b. This is called the Schwarz inequality, and the reader is asked to 
prove it in Problem 2-11. Two vectors are said to be orthogonal if 
a'b — 0. 


EUCLIDEAN SPACE: An n-dimensional euclidean space, symbolized by 
E^, is defined as the collection of all vectors (points) a = lag. а) 
For these vectors, addition and multiplication by a scalar are defined by 
the rules for matrix operations. Furthermore, associated with any two 
vectors in the collection is a non-negative number, called the distance be- 
tween two vectors; the distance is given by (2-60). 


Here we have a definition of the n-dimensional spaces that we shall 
be using. The familiar two- and three-dimensional spaces of elementary 
geometry were euclidean spaces, 


2-9 Linear dependence. A vector a from E" is said to be a linear combi- 
nation of the vectors a;,..., ay from E" if a can be written 


а = Nia; + Хаз +--+ + уа, (2-53) 
for some set of scalars X. 


LINEAR DEPENDENCE: А set of vectors 81,..., am from Е" is said to be 
linearly dependent if there exist scalars М not all zero such that 


Mai + №аз + +++ + Anam = 0. (2-54) 


Tf the only set of X; for which (2-54) holds is N= Ag = +--+ = Ay = 0, 
then the vectors are said to be linearly independent. 


A set of vectors which is not linearly dependent must be linearly inde- 
pendent. If we let aj = [a11,..., dail)... , am = [aym,... › апт), then 
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on writing out the component form of (2-54), we obtain 


ап + arada +" + 015. = 0, 
B (2-55) 
anı + anada + +++ + ans = 0, 


which is a set of n simultaneous linear equations in m unknowns, the А;. 
If there exists a solution to this set of equations with not all A; = 0, 
the given vectors are linearly dependent. Later we shall study systems of 
equations like (2-55). 

Linear dependence and independence are properties of sets of vectors, 
and not of the individual vectors in the set, For a set containing a single 
vector a, the definition of linear dependence says that a is linearly de- 
pendent if there exists a А ~ 0 such that Xa = 0. Thus the set containing 
a single vector is linearly independent if a # 0, and linearly dependent 
if a = 0. 

If a set of vectors аџ,... , am from E" contains two or more vectors, 
then the set is linearly dependent if and only if some one of the vectors is 
a linear combination of the others. To prove this suppose that one of 
the vectors, say am, can be written as a linear combination of the 


others, i.e., PIS. ih 
ал = ЎЎ ма ог У) ма; — аһ = 0. (2-56) 
peri ici 


Consequently, the vectors are linearly dependent. On the other hand, 
if the vectors are linearly dependent, then (2-54) holds, and at least one 
№ = 0. Suppose Am # 0. Thus we сап write 


mol). 
dale tl y aT (2-57) 


and one of the vectors can be written as a linear combination of the 
others. 

A vector a is said to be linearly dependent on a set of vectors a1, . . . , âm 
if a can be written as a linear combination of ai, . . . ‚ат; otherwise, a is 
said to be linearly independent of а, . . . , ат. It can be noted immediately 
that the null vector is not linearly independent of any other vector or set 


of vectors since 


0 = 0a, + Oaz +++: + Oam- (2-58) 


No set of linearly independent vectors can contain the null vector. 

It is clear that if a set of vectors is linearly independent, then any 
subset of these vectors is also linearly independent. Similarly, if any 
given set of vectors is linearly dependent, any larger set of vectors con- 
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taining the given set of vectors is also linearly dependent. We say that 
the maximum number of linearly independent vectors in a set is k if in 
the set there is at least one subset of k vectors which are linearly inde- 
pendent, and if there is no linearly independent subset containing k + 1 
vectors. 

Suppose that k « m is the maximum number of linearly independent 
vectors in a set of m vectors a1, .. ., am from E". Then, given any linearly 
independent subset of k vectors in this set, every other vector in the set 
can be written as a linear combination of these k vectors. To see this 
label the vectors so that 8;,..., a; are linearly independent. The set 
81,...,8,, а, must be linearly dependent for any r= k + 1,...,m. 
This implies that (2-54) holds and at least one А; #0. However, A, 
cannot be zero, because this would contradict the fact that aj,...,&k 
are linearly independent. Hence a, can be written as a linear combination 
Of 81,..., а. 


ExaMPLEs: (1) The unit vectors for E", eı, ..., e, are linearly inde- 
pendent since 
n 
у) Ме; = [x А, о, M] = 0 


ici 
implies that à; = А = ++: = An = 0. 


(2) Any two non-null vectors a, b are linearly dependent if Х,а + 
Agb = 0, dy, ào Æ 0 (why must both A; and As differ from zero?), or 
b = Ла. Thus, two vectors are linearly dependent if one is a scalar multiple 
of the other. Geometrically, in 7%, this means that two vectors are 
linearly dependent if they lie on the same line through the origin. Be- 
cause only vectors lying along the same line are linearly dependent in 
E?, it follows that any two vectors not lying along the same line in E? 
are linearly independent. In E?, any three vectors are linearly dependent 


22 


х = Ма + № 
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since one of the vectors can be written as a linear combination of the 
other two (see Fig. 2-3). 

In E?, any two vectors lie in a plane through the origin. Any vector 
which is a linear combination of these vectors will lie in this plane. Thus, 
if three vectors in E? are linearly dependent, they lie in а plane through 
the origin. Any three vectors in E? which do not lie in a plane through 
the origin are linearly independent. Any four vectors im E? must be 
linearly dependent. 


2-10 Bases. The concept of a basis is very important in linear pro- 
gramming. We begin with the following definitions: 


SPANNING SET: A set of vectors &,...,8 from E" is said to span or 
generate E" if every vector in E" сап be written as a linear combination 
Of aj, ... , 8r. 


From (2-49) we note that the unit veetors span E^. A spanning set 
containing the smallest possible number of veetors must be linearly 
independent, since in a linearly dependent set at least one vector can be 
written as a linear combination of the others and hence is not needed in 
the spanning set. 


Basis: A basis for Е" is a linearly independent subset of vectors from Е" 
which spans the entire space. 


The unit vectors are a basis for E" since they are linearly independent. 
A basis for E" is by no means unique. As we shall see, there exists an 
infinite number of bases for E". 

The representation of any vector in terms of a set, of basis vectors is unique, 
that is, any vector in E" can be written as a linear combination of a set - 
of basis vectors in only one way. To prove this, let b be any vector in E^, 
and let aj, . .. , a, be a set of basis vectors. Suppose that b can be written 
as a linear combination of the basis vectors in two different ways, namely; 


E : 
ъ= ra; b=), Nar. (2-59) 
tel i=l 
Subtracting, we obtain У; (м — Ма; = 0, and since the set of basis 
vectors is linearly independent, М — M = 0, = 1,...,7, огу = №, 


We shall now investigate the conditions under which an arbitrary 
vector b from E" can replace one of the vectors in a basis so that the 
new set of vectors is also a basis. The technique of replacing one vector 
in a basis by another such that the new set is also a basis is fundamental 
to the simplex technique for solving linear programming problems. 

Given а set of basis vectors аз, .+ , & for Е" and any other vector b # 0 
from E^. Then in the expression of b аз a linear combination of the ai, 
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yy aab (2-60) 


i=l 


if any vector a; for which œ; # 0 is removed from the set a1, .. . , a, and b 
is added to the set, the new collection of r vectors is also a basis for E". To 
prove this, note that at least one a; in (2-60) is different from zero since 
Ъ +“ 0. Number the vectors so that a, # 0. We wish to show that the 


set 8,,..., 8, 1, b is a basis for E". To show that the set 81, ..., &—1, 
b is linearly independent, suppose the contrary, i.e., assume that 

7-1 

У) ёа: + b = 0, (2-61) 


and at least one à; or à is not zero. Now 6 cannot be zero in (2-61) because 
this would contradict the fact that the aj, 7 = 1,...,7 — 1, are linearly 
independent. However, using (2-60) to eliminate b, we obtain 


r1 
D (6: + ада; + боа, = 0, (2-62) 
i=l 
and êa, ~ 0. However, this contradicts the fact that the a;, 7 = 1,..., 7, 
are linearly independent. Hence the set a;,...,8, 1, b is linearly 
independent. 

It is easy to see that any vector x in E" can be written as a linear corabi- 
nation of а;,...,а,_1, b. Since the a; i = 1,..., 7, form a basis, we 
can write 

xc» Tar (2-63) 
i=l 


Eliminating a, in (2-63) by means of (2-60), we obtain 


т—1 
Qi Yr 
is шн jd E -64 
х z Siv, ) ai + 2b, (2-64) 
which expresses x as a linear combination of a;,...,8, :, b. Hence 
81,..., Arı, b is a basis for E". 


If in (2-60) b replaces a vector a; for which a; = 0, then the new set 
of vectors is linearly dependent and does not form a basis for E". To 
prove this, take o, = 0 and replace a, by b. Then 


r—1 
b> >) aa = 0. (2-65) 


i=l 


The vectors @1,..., ағ, b are linearly dependent. 
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13 


T, 


Figure 2-4 


ExaMPLE: Imagine that we wish to insert the vector b = [3, 0, 4] 
into the basis е, ез, ез and remove one of the vectors in the basis. We 
have b = Зе, + Оез + 4ез. According to the preceding discussion, we 
can remove either e, or ез to obtain a new basis; that is, ез, es, b or 
ei, ез, b will form a basis for Е°. We cannot remove the vector е; and 
still maintain a basis. This is illustrated geometrically in Fig. 2-4. If 
e, is removed, е, es, b all lie in the тутз-Ріапе and are linearly dependent. 
If either e; or e; is removed, the new set of vectors does not lie in a plane. 


2-11 Number of vectors in a basis for E". The results obtained above 
lead us to suspect that every basis for E^ contains precisely п vectors. 
This is correct, as we shall now demonstrate. " 

We first prove that any two bases for E" have the same number of 
basis vectors. Let 81,...,8u be one set of basis vectors for E" and 
bi, ..., by be any other set of basis vectors. We wish to show that u = v. 
We can number the vectors a; so that when b, is expressed as а linear 
combination of the a; By = Хаз, then Мм #0. Hence by the results 
of the previous section, а,...,й«—1, b, is a basis for Е". Now if we 
write eet 

byt = Dy biti + Obs (2-66) 
i-i 


at least one ô; must be nonzero, for otherwise the set of b; would not be 


linearly independent. Take Bui #0; then Ar >, 8u-2 by-1, by is 
a basis for E^. This process can be continued until a basis is found which 


must take one of the following two forms: 


ашышы Scribit QUE 7 by eero Por 


ә 
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There must be at least ая many a; as by, Otherwise a state would be 
reached where one b; was a linear combination of some of the others, 
contradicting the fact that the by are linearly independent. Hence u > v. 
However, one can start with the b, and insert the a, This would yield 
s > u. Therefore и = v, and we have proved the correctness of our 
assertion 


To determine the actual number of vectors in a basis for E", it is only 
necessary to find a basis and count the number of vectors in this basis, 
We have already shown that the n unit vectors form a basis for Æ", It 
immediately follows that every basis for E" contains precisely n vectors. 

We сап now see that any set of n linearly independent vectors from Е" 
forms а basis for E". Let &,,. . . , & be such a set of linearly independent 
vectors, To prove this result, we only need to start with the basis 
$),..., €. and insert the а. After n insertions, which can always be 
made (for the same reasons used in proving that two bases always have 
the same number of elements), we obtain а new basis aj,...,a,. Thus 
it is obvious that any n + 1 vectors from E" are linearly dependent. 

Any т < n linearly independent. vectors from E" form part of а basis 

for E", that is, if ay,...,%_ from E" are linearly independent, then 
т — m additional vectors in E* can be found such that a, . . . , an and 
the other à — m vectors in E" form а basis for E". Again, the proof 
depends on starting with a basis and inserting the а, in precisely the 
name way as was done before. If we start with unit vectors as a basis, 
we вее that the additional n — m vectors used to fill out the basis can 
be unit vectors, Ax a specific example of this theorem, any non-null 
vector from E is part of a basis, and therefore there is an infinite num- 
ber of bases for E", 
Ме have characterized the dimension n of a euclidean space by the 
‘number of components in the set of vectors which form the space, that 
is, В" in the sot of all n-component vectors. We have shown that the 
maximum number of linearly independent vectors in Е" ія n. In certain 
Tenpecta, it ія more desirable to define the dimension of a space as the 
maximum number of linearly independent vectors in the space. This 
definition avoids connecting the dimension of a space to the number of 
components in the vectors fi the space. The advantage of this 
spproach will become clear in next section. 


2-12 Vector spaces and subspaces. 
VECTOR APACE! A vector space is a collection of vectors which is closed. 
under the operations of addition and multiplication by a scalar. 
The expression “a set of vectors is closed under addition and multiplication _ 
by a scalar” means that if a, b are in the collection, the sum a -+ b is also 
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in the collection, and if a is in the collection, M is in the collection for 


any scalar À. 

Note that distance need not be defined in a vector apace, Also note 
that 0 ix always an element of a vector space, and so ія —а if a is. The 
totality of all n-component vectors is called an n-dimensional vector 
space and is denoted by Va. Although Б" is clearly a vector space, it 
does not follow that any Va is an E". " V» à 

Sunspace: A subspace Sy of Va is defined to be a subset of V which ts 

itself a vector space. 

The subscript n on S, means that the vectors have n components, Le, 
are elements of Va. We define the dimension of S, to be the maximum 
number of linearly independent vectors in Sa, We consider S, = V, 
to be a subspace of V, having dimension n, Le., the subspace is all of Va. 


Examece: The set of vectors in B® lying in the plane x» = zy it a 
subspace of E’, ie., the set of all vectors (21, та, zal (proof), ‘Now Е 


[ri 22, ta] = де, + 2200, 1, 1), 


and the dimension of the subspace is two, This is precisely what we want 
intuitively, since the vectors lie in a plane, ie, in a two-dimensional 


space, 
A clear geometric interpretation of subspaces in А? can be given. 
Any subspace of E? is either E? itself, в plane through the origin, а line 
through the origin, or just the origin itself (this is a subspace of a single 
element and has dimension 0), xil 
A set of vectors from Sq is said to span or generate $, if every veetor 
in S, can be written ns a linear combination of this set of 
linearly independent set of vectors from S, which spans S, 
basin for S,. If S, 4 dimension k, then a basis for Sù will contain pre- 
cisely k vectors, If Sq ія a subspace of Е" having dimension А, then, 
except for notation, S, is the sume as B. 


1 


2-13 Rank. The n columns of an m X т matrix A ean be considered 

to be vectors in E". Recall that the columns of A are denoted by а, 
Rann: The rank (or more precisely Uv column rank) af an m X n matriz 
A, written r(A), is the mazimum number of linearly independent columna 
in А. 


Fxarux: The mnk à ap benda Miya law sie T 
(ei... , €n), and the e, are linearly "cR 

It can be shown [LA 4-4) that the rank of A is А every 
minor of A having order k + 1 vanisbes, while there is at least one minor 


: 


ч ag - 4 
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of order k which does not vanish. This theorem provides a way to de- 


termine the rank of a matrix. We determine the order of the largest ' 


nonvanishing minor in A, and this is the rank of A. Although this method 
for determining the rank of a matrix is not too efficient computationally, 
it is of considerable theoretical importance. By means of this theorem 
it is also possible to find the maximum number of linearly independent 


vectors in a set aj, .. . , a, from E^, We form the matrix A = (a),..., a) 
and determine the order of the largest nonvanishing minor. " 
EXAMPLE: 
270 7 
Ais) 3:3 4:6, 
224 


has rank 2, since |A| = 0, but there are several minors of order 2 which й 
do not vanish. 


е row tank of a matrix А ean be defined as the maximum number of 
, linearly independent rows in A. It is easy to show that the row rank of A 
“is equal to the column rank, i.e., the maximum number of linearly inde- 
pendent columns in A is equal to the maximum number of linearlyyin= - 
dependent rows, The rank of A is thus unique, and we need not distinguish 
between the row rank and column rank. To prove this it is only necessary 
to note that the row rank of A is equal to the column rank of A’, and 
that the largest nonvanishing minor in A appears in A’, and conversely. 
The above results tell us that the columns (or rows) of an nth-order 
nonsingular matrix are linearly independent and form a basis for E". 
Also, an nth-order matrix whose columns (or rows) are a basis for Æ" 
is nonsingular, 
"The following results are sometimes useful (LA 4-3]: 
(1) (AB) < min [r(A), r(B)]. (2-67) 
(2) If a matrix of rank k is multiplied in either order by a nonsingular 
matrix, the rank of the produet is k. 


2-14 Product form of the inverse. In the simplex method we have 
occasion to compute the inverse of a matrix for which only one column 
is different from that of a matrix whose inverse is known. Suppose that 
we have a nonsingular matrix В = (b;,..., b,) and that B^! is known. 
Now we replace column r of B with a and wish to compute the inverse of 

a = (bi... bya, a, byyi,...,b,). If the columns of B and B, are 
viewed as two bases for E^, then B, is obtained from B by changing a 
single vector in the basis. Hence the results of Section 2-10 may be 


^ 
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ET d 
applied. Thus, if a = Dy убу, we-know that B, will be nonsingular if 
and only if y, # 0. Assuming this to be the case, we note that 


@. b, — Ban, n 
where 1 t 
" T 
“= [- и... 2,1, Кыр аы}, (2-68) 
, Yr Yr Ur уг. Ur. 
Hence, 
B = B,E, E (e... Cross, Opty «+ >On) (2-69) 
or 


52) = EBT". (2-70) 


Note that Е is obtained from the identity matrix by replacing the ss 
column with эу. Also, to compute у we only need y = B-'a. 

The above theory provides another way of computing the inVeree of " 
nonsingular matrix В = (bi, ..., b,). We begin with the identity matrix 
(I7! — I) and insert the columns of B one at a time. For example, we 
might begin by inserting the first column of B into I, removing column 
1 of I, to yield B,. Then we might insert the second column of B into 
By removing column 2 of By, to yield By, ete. Then 


\ B7 Бүгүн: EQ *(2-71) 
' where 
E; = (e, 65 ..., 6i-i mo Citt 68); (2-72) 
Yii yicaa d Vitii Voi 
e| 1 йш o, Шын, ey CT IY E 
9 [ Yii Yii Yii Yii Vii m 
and 
yi = Bilb, | Вг! = Е@—,...,Е, = ЕВ, t= 1,...,п; 
By! = 1 (2-74) 


We call (2-71) a product form of the inverse. Form (2-71) is not unique 
since different orders of insertion of the columns of B may be used and 
may indeed be required. 


ExaAMPLE: Consider a matrix B = (Ь,, bs, ba) whose inverse is 
204 
В-! = 5 (116: 
7,9 3 
Let us replace the second column of B with a = [1,8,6] to yield the 
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matrix B, = (bı, a, b3). To compute By! we first compute 
y = B'a = (26, 49, 97]; 
then we form the vector 
yı 1 Va 26 1 97]. 
»-[- Ua Y2 اھ‎ 49’ 49’ - |: 


next we construct the matrix E by replacing the second column of 
with m, ie., 


1 —#§ 0 
E=|0 æ 0l: 
ре 
Finally, 


2-15 Simultaneous linear equations. A set (or system) of m simi 
taneous linear equations in n unknowns, 7ı, ... х, has the form 


ах d +++ + Ginta = bi, 
$ (2-7 
Omit + ата = Dy, 


where the a,j, b; are known constants. If we write A = la;jl, x 
[i tal, bm [b ..- bna], then using matrix notation, we see tl 
(2-75) becomes simply Ax — b. 

Let us study for the moment the case where m = л. Then if r(A) = 
so that |A| = 0, we obtain the unique solution x = A™b. The soluti 
is unique since the inverse is unique. We shall discuss later what hap 
if r(A) < n. Using (2-30) and (2-33), we see that the solution x = A1 
can be, written in component form as 


1 «€ : 
ж = Аг У) Ауфу i-—1l...,m. (2-71 
АІ e 


From the definition of a determinant, it will be observed that У; Ау , 
is the determinant of the matrix formed from А by replacing column 1 
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with b. Thus 
А bi 412... Qin i азу...) by 
ESTATES E SON ta ДАГ]. Н 
bn аз... Onn dpt.» On m—1 On 


(2-77) 


Equation (2-77) is called Cramer's rule for finding the solution to a system 
of n equations in n unknowns. It is not an efficient numerical procedure 
for finding solutions, but it is of considerable theoretical value. 

A fairly efficient numerical procedure for solying a system of n equations 
in n unknowns is called the gaussian reduction scheme. It is perhaps 
the first thing one might think of in attempting to solve such a set of 
equations, Number the variables and equations in (2-75) (with m = n) 
so that ayı »* 0: Divide the first equation by arı and use the result to 
eliminate тү in equations 2,...,n. This yields 


ay + data + +++ + был = by, 
зата + 5-5 + бани = Da, 
: (2-78) 
` Anata +++ + был» = n, 
where 
ay = 20, j —25,...,^j du = ау — San, 
ыз}! yn) Ч] 
and 


bi En Te 
b, au , by b; ai n t 2, eec, Th 


Now divide the second equation of (2-78) by da» (assume equations and 
variables are renumbered, if necessary, so that 232 »* 0) and eliminate 
za in equations 3,...,7. In a finite number of steps we arrive at a set 
of equations having the form (provided |A| »* 0) 


1 Аа... hin || zi т 
dE Ga ре. (2-80) 
1 хл In 


Then z, = gn, and by back substitution 23.) = gn—1 — Mn—1.ndn, ete. 


52 MATHEMATICAL BACKGROUND {снар. 2 


Now let us return to (2-75) and consider the general ease where m can 
differ from n. We allow m to be >, =, «n. To begin we wish to determine 
the conditions under which we know that there exists at least one solution 
to (2-75). In order to do this, we first introduce the m X (n + 1) matrix 
A, = (A, b); A, is called the augmented matrix for the system and is 
formed by annexing to A the vector b which becomes column n -+ 1, 
Tt will be noted that r(A») > r(A) since every minor in A also appears 
in Ay. It is now easy to see that if r(A,) > r(A), then there does not 
exist а solution to (2-75). This follows since if r(A,) = k > r(A), every 
set of k linearly independent columns from A, must contain b. Hence 
b cannot be written as a linear combination of the columns of A, i.e, І 
there do not exist xj such that Уу туа; = b. On the other hand, if 
r(Ay) = r(A) = k, then there exists a set of k columns from A such that 
every column in А», and in particular b, can be written as a linear combi- 
nation of these k columns. Hence there exists at least one solution to (2-75) 
and, in fact, a solution with no more than k of the variables different from 
zero. The set of equations (2-75) is called consistent if there is at least 
one solution; otherwise it is said to be inconsistent. 


ExaMPLE: The equations 
3z, + 4x, = 7, 
2.2551 + 325 = 1 


are inconsistent, and there is no solution since r(A) = 1 and r(Ay) = 2, 
If the first equation is multiplied by 3, the left-hand side becomes the 
left-hand side of the second equation. However, the right-hand side of 
the first equation does not become the right-hand side of the second 
equation. Geometrically, the equations represent two parallel lines which 
do not intersect, ) 


Let us now study a system of equations of the form (2-75) for the 
case where r(A) = r(Aj) = k < m. To be specific, number the equa- 
lions so that the first k rows of the matrix A are linearly independent. 
Then, denoting the rows of A, by (a, bj), one can write any row with 
index r > k as a linear combination of the first k rows, ie., 


k 
@',Ь) = Уа), =r =k 4-1,...,m, (2-81) 
=1 
or 


Г k 
a = Da; = Mb rk +1,... A (2-82) 


fowl f-1 


Now suppose that x satisfies the first k equations of (2-75), i.e., ax = by, 
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f= 1,..,, k, Then 


L] LI 
ax Уак = Уна, т=Е+1,...,т. (2-83) 
i=l tel 

Thus any x which satisfies k equations ax = b, for which the corre- 
sponding rows of A are linearly independent satisfies all m equations. 
In other words, all but k equations may be ignored in the process of 
seeking solutions to (2-75). We say that m — k of the equations are 
redundant, Le. they do not place any additional constraints on the 
variables. 

We have assumed that the first Ё rows of the matrix A are linearly 
independent. Denote by A, the submatrix formed from the first k rows 
and columns of A. ‘Then we can write the first k equations of (2-75) as 


Aix, + Ris = Б", (2-84) 


where x, = [zy ..., zi], X6 lian +2) ta), b* = (by, «1; bal, and 
R contains the last n — К columns of the first А rows of A. Any x = 
[x.,, ху] which satisfies (2-84) satisfies all m equations (2-75). Now nssume 
that the variables have been numbered so that Ay is nonsingular; then we 
can write 

x, = Aj'b* — Ат!Вх,. (2-85) 


Hence given any xy, we can solve uniquely for x, in terms of x», There- 
fore, arbitrary values can be assigned to the n — k variables in xy; values 
for the remaining k variables in x can be found, во that x = [хх] is a 
solution to (2-75). All solutions to (2-75) сап be generated by assigning 
all possible values to the set of variables in xp. Thus, ifn > &, there will 
be an infinite number of different solutions to (2-75). We have shown, 
therefore, that a set of equations Ax = b has either no solution, a unique 
solution, or an infinite number of solutions, 

Let x; be any solution to (2-75). Then any other solution xg to (2-75) 
сап be written xy == xy + y, y = Xs — Xi. Note that y satisfies Ay = 0, 
А set of linear equations Ax = 0 is said to be homogeneous. We have 
shown that every solution to (2-75) can be obtained from one solution 
to (2-75) plus all solutions to the set of homogeneous equations, It is 
casy to demonstrate that the set of all solutions to Ах = 0 is а subspace 
of E". The dimension of this subspace can be shown to be n — k, where 
r(A) = k [LA 5-6]. Then the dimension of the space generated by the 
solutions to (2-75) must also have dimension n — k. However, if b # 0, 
the space is not a subspace of Е" because 0 is not a solution. The space 
generated by the solution to (2-75) with b » 0 is called an affine sub- 
space. This space is translated away from the origin because b »* 0. 
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EXAMPLE: The solutions to axı + azto + agz4 = b lie on a plane. 
This plane does not pass through the origin unless b = 0. 


2-16 Basic solutions. We now wish to study solutions to a set of m. 
equations Ax = b in n > m unknowns, which have as many of the 
variables equal to zero as possible. From the previous section we know 
that if r(A) = k and we select any k linearly independent columns from A, 
we can assign arbitrary values to the л — k variables not associated 
with these columns, The remaining k variables will be uniquely determined 
in terms of the n — k variables. Thus for such a system, we can set 
^ — k variables to zero; the remaining k variables will, in general, be 
different from zero since the equations must be satisfied (in certain cases, 
however, one or more of these k variables will be zero). To be specific, 
we shall assume that r(A) = r(A,) = m. This implies that none of the 
equations is redundant. 


Basic SOLUTION: Given a. system of m simultaneous linear equations in 
n unknowns, Ax = b (m < n) and r(A) = m. If any m X m non- 
singular matriz is chosen from A, and if all the n — m variables not 
associated with the columns of this matrix are set equal to zero, the solution 
lo the resulting system of equations is called a basic solution. The m vari- 
ables which can be different from zero are called basic variables. 


Let us choose m linearly independent columns from A and form a 
matrix B whose columns are the columns selected from A. Then we can 
write a basic solution as xg = B~'b, where all variables not associated 
with the columns from A in B are set equal to zero. 

How many basic solutions are possible in a system of m equations and 
n unknowns? The. answer is the number of combinations of n things 
taken m at a time, that is, n!/m!(n — m)!, and this is the maximum 
number of basic solutions. To obtain all these Solutions, every set of 
m columns from A would have to be linearly independent. 


DEGENERACY: A basic solution to Ax = b is degenerate if one or more 
of the basic variables vanish. 


It is easy to prove the following {LA 5-8]: 

(1) A necessary and sufficient condition for the existence and non- 
degeneracy of all possible basic solutions of Ax = b is the linear inde- 
pendence of every set of m columns from the augmented matrix A, — (A, b). 

(2) A necessary and sufficient condition for any given basic solution 
xg = В-'Ь to be nondegenerate is the linear independence of b and 
every set of m — 1 columns from B. 

If we have a solution to (2-75) which involves precisely m nonzero 
variables, and if this solution is unique, then it must be a basie solution. 
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To prove this it is only necessary to show that the columns of A associated 
with the nonzero variables are linearly independent. If they were de- 
pendent, then some one of them, say k, could be expressed as a linear 
combination of the others. This would, however, contradict the fact 
that the solution is unique because, for any arbitrary value of хь, values 
of the other m — 1 variables could be found (the zero variables remaining 
zero) that maintain a solution. 


ExAMPLE: Let us find all basic solutions for 


Tı + 250 +23 = 4, 
221 + za + 5z3 = 5. 


The maximum possible number of basic solutions js 31/211! = 3. 

First we set 23 = 0 and solve for xı and тз, obtaining ху = 2, 25 = 1. 
Next we set x2 = 0 and solve for a, and zs, obtaining жу = 5,23 = —1. 
Finally, we set тү = 0 and solve for za and zs, obtaining x = $, t3 = $. 
Thus all three basic solutions exist and are nondegenerate. 


2-17 Linear transformations. The algebra of matrices is often referred ' 
to as the algebra of linear transformations. Let A be an m X n matrix. 
Then for any vector x in E", the vector y = Ax can be considered to 
be a vector in E". To each vector or point x in E" there corresponds 
a point y = Ах in E". The point y = Ax is called the image of x. We 
say that A maps E" into all or part of E", The relation y = Ax is also 
said to transform each point in 2” into a point іп E". A matrix trans- 
formation of the form y — Ax is said to be a linear transformation, A 
matrix or linear transformation has the property of preserving addition 
under the transformation: If y; = Ax; and yz = Ах», then if уз = 
A(x, + хо), it follows that yg = yı + Yo. Also a matrix or linear trans- 
formation preserves multiplication by a scalar, ie, if y = Ax and f = 
A(x), then $ = Ay. 

If A is an m X r matrix and B isan r X n matrix, the relation y = ABx 
can be viewed as a sequence of linear transformations. The matrix B 
takes a point in E" into a point in £”, and A takes the point in Æ” into 
a point in E". 


2-18 Point sets. On various occasions, we have used the notion of n 
set, for example, a set of n component vectors. The notion of a set is so 
basic that it is somewhat difficult to give its definition in terms of more 
fundamental ideas. The following expressions are synonymous: (1) a set 
of elements, (2) a collection of objects, (3) а number of things. A set 
consists of a finite or infinite number of elements. Sets will be denoted 
by capital letters, for example, A, B. The elements of the set will be 
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denoted by a; b; ete. Braces { } enclose the elements belonging to a 
set. Thus the set A containing elements a; can be written A — {ar}. 

Two sets A and B are equal, A = B, if they contain precisely the 
same elements. The notation a; € A indicates that a; is an element 
of A, and b; @ A means that b; is not an element of А. A subset B of a 
set A is a set all of whose elements are in A. However, not all elements 
of A need to be in the subset B; B isa proper subset of A if A contains 
at least one element which is not in B. The notation B C A or A DB 
indicates that B is a subset of A. The intersection of a finite number of 
sels A,,..., Ax, written A1 ПАП: N Aj, is the set containing all 
elements common to Ay,..., Az. The union of a finite number of sets 
Ар... Ap, written Ay U Aa U-++U As, ds the set containing all elements 
in at least one of the sets Ay,..., Ax. 

Point sets are sets whose elements are points or vectors іп E". Since 
our approach here will be mainly geometric, we shall often refer to vectors 
as points in E^. Our future discussions will be limited to point sets. These 
sets may contain either a finite or infinite number of elements. Usually, 
however, they will:contain an infinite number of elements. Point sets 
are often defined by some property or properties which the set of points 
satisfy. In E?, for example, let us consider the set of points lying inside 
а circle of unit radius with center at the origin, that is, the set of points 
satisfying the inequality її +23 < 1. A convenient representation for 
the set X is X = [[rj, zale? + x? « 1]. In general, the notation 


X = {x|P(x)} (2-86) 


will indicate that the set of points X = {x} has the property (or prop- 
erties) P(x). In the above example, the property P is the inequality 
xi + 23 < 1. If there are no points with property P, then the set (2-86) 
contains no elements, and is called empty or vacuous. We shall always 
assume that there is at least one element in a set unless otherwise stated. 

A hypersphere in E" with center at a and radius € > 0 is defined to be 
the set of points X = {x||x — a| = e], i.e., the equation for a hypersphere 
in E" is 33-1 (5; — aj? = e?, which isa circle in E? and a sphere in J3. 
The inside of a hypersphere with center at а and radius є is the set of points 
X = (xix —a| < e}. An €neighborhood about the point a is defined 
as the set of points lying inside the hypersphere with center at a and radius €. 
A point a is an interior point of the set A if there exists an €-neighborhood 
about a which contains only points of the set A. An interior point of A 
must be an element of A. A point a is a boundary point of the set A if 
every €-neighborhood about a (regardless of how small € > 0 may be) contains 
points which are in the set and points which are not in the set. A boundary 
point of A does not have to be an element of A. A set A is an open set 
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if it contains only interior points. A set A is a closed set if it contains all 
its boundary points. A set may be neither open nor closed, i.e., it may 
contain some but not all of its boundary points. A set is said to be strictly 
bounded if there exists a positive number r such that for every a є A, |a| < т. 
A strictly bounded set lies inside a hypersphere of radius r with center 
at the origin. The set A is bounded from below if there exists an x with each 
component finite such that for all a & A,r < a. A set which is bounded 
from below has a lower limit on each component of every point in the set. 


2-19 Lines and hyperplanes. Consider the line in E? shown in Fig. 2-5. 
This line passes through the points х; and хз, and the vector x? — x, 
is parallel to the line. It follows from the law for addition of vectors 
that for any point x on the line, there exists a А such that x = xi 
A(x; — xi) = Axa + (1 — Ах. In E", we define the line through the 
two points xy and X2, х 5 хә, to be the set of points 


= [x|x = Ах, + (1 — xy, all real ^]. (2-87) 

In Fig. 2-5 note that as \ varies between 0 and 1, that part of the line 

joining x; to xs is traced out. In E", the line segment joining points x, 
and xa is defined to be the set of points 

X = [xix = X; + (1 — Ах, 0 & XA 11. (2-88) 


In B®, суху + сәлә = z is a line for given values of сү, c», and z, while 
in ЕЗ, сіх + сото + cara = z is а plane. In E" we say that the set of 


Figure 2-5 
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points x satisfying 
citi сота b cb сата = 2 (not alle — 0) (2-89) 3 


defines a hyperplane for given values of the c; апй г. If we write c= 
(ci,...,¢€,), then the hyperplane (2-89) can be written cx = z. Ina - 
linear programming problem the set of all x yielding a given value of the _ 
objective function is a hyperplane. 

A hyperplane passes through the origin if and only if z= 0. Thus a 
hyperplane through the origin can be written cx — 0. Note that this 
relation implies that ¢ is orthogonal to every vector x in the hyperplane. 
When z з 0, then if x;, хз satisfy (2-89), i.e., lie on the hyperplane, 
C(x — xj) = 0. Thus, in general c is called à vector normal to the 
hyperplane; +¢/|c| are called unit normals. Two hyperplanes are said 
to be parallel if they have the same unit normal. Thus the hyperplanes 
Сух = 2; and сәх = zz are parallel if c, = №2, А ғ 0. 

Consider the hyperplane сх = zy. Then consider the set of points 
Xo + de’, А > 0, схо = zo (c is a row vector, so c' is a column vector). 
Note that these points lie on the hyperplane cx — 2;, where 21 = zo + 
Меј? > zo. Points lying on the hyperplane cx = 21 satisfy cx > zo. 
Intuitively we say that the hyperplane cx = z, is obtained by moying 
€x = гу parallel to itself in the direction of c. 

A hyperplane cx = z divides all of Е" into three mutually exclusive 
and collectively exhaustive sets. These are: (1) X, = {х|сх < z}, 
(2) Хә = [x|ex = z], (3) Xs = [xlex > г}. The sets X3, Хз are called 
open half-spaces. In E? and Е?, a half-space is all of E? or E? lying on 
one side of a line or plane, respectively. The sets X, = {x|ex < z} and 
Xs = {xlex > z} dre called closed half-spaces. Note that X, X 5= Xa. 

Hyperplanes are closed sets. In fact, every point on the hyperplane is a 
boundary point, and no point not on the hyperplane can be a boundary 
point. Also, closed half-spaces are closed sets. On the other hand, open 
half-spaces are open sets, 


2-20 Convex sets. We begin by defining some important concepts. 


Convex set: A set X is conver 1f for any points xi, хь in the set, the’ 
line segment joining these points is also in the set. 


This definition implies that if Xi, X2 € X, then every point x = №» + 
(1 — Xxi, 0 € ^ < 1, must also be in the set. By convention, we say 
that any set containing only one point is convex. The expression Ах: + 
(1 — Xx, 0 <А < 1, is often referred to as a convex combination of 
Xi, ха (for a given А). Intuitively, a convex set cannot have any “holes” 
in it; it is “solid,” and not “re-entrant,” i.e., its boundaries are always 
“flat” or “bent away” from the set. 
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ExTREME POINT: A point x is an extreme point of a convex set if and 
only if there do not exist other points Xy, Xo, Xy 7é Xo, in the set such 
that x = Axa + (1 = Ax, 0 <A <1. 


Note that strict inequalities are imposed on A, The definition stipulates 
that an extreme point cannot be “between” any other two points of the 
set. Clearly, an extreme point is a boundary point of the set, for other- 
wise it would be between some other two points. Not all boundary points 
of a convex set are necessarily extreme points. Some boundary points 
may lie between two other boundary points. 


ExawPLES: (1) A triangle and its interior form a convex set. The 
vertices of the triangle are its only extreme points. 

(2) The set X = [[r; 22122 + z2 € 1] is convex. Every point on 
the circumference is an extreme point. 

(3) The set in Fig. 2-6 is not convex since the line joining x; and xg 
does not lie entirely within the set. The set is re-entrant. 

(4) The polygons in Fig. 2-7 are convex sets, and the extreme points 
are the vertices. Point x; is not an extreme point because it can be repre- 
sented as a convex combination of xg and xa, with 0 < A < 1. 


A hyperplane is a convex set. To prove this it is only necessary to note 
that if x, and хз are on the hyperplane, that is, cx; = z, cx, = 2, then 
x = Mg + (1 — Xx, is on the hyperplane since 


сх = cxt; + (1 — Nx] = Асх; + (1 — Alex, = AZ + (1 — 2 = 2. 
(2-90) 
Similarly, open and closed half-spaces are convex sets. 

The intersection of two convex sels is also convex. To prove this consider 
the convex sets X,, Хз and let хі, x2 be any two points in Хз = X, N X2 
(if there is only one point in Хз, then X is automatically convex). Thus 
хә + (1 — № € X, for 0 € A € 1 and Ax; + (1 — Xx, € Хз for 


E A 
e c) 


Fieure 2-6 Figure 2-7 
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0.5 л < 1. Hence Ах: + (1 — Ахь € X; п X2 = Xz, and Xs is con- i 
vex. If X, and X; are closed seis, then Хз = X4 N Хә is also closed. 
To see this, it is only necessary to note that every boundary point of X3 | 
is a boundary point of X, or X2. However, X;, Хә are closed; hence Xs 
contains all its boundary points and is closed. More generally the inter- 
section of any finite number of convex sets is convex, and if each of the sets 
18 closed, the intersection is also closed. 

The above results show that the intersection of a finite number of hyper- 
planes, or half-spaces, or of both is a convex set. Furthermore, the inter- 
section of a finite number of hyperplanes, or closed half-spaces, or of both 
is a closed convex set. This result has an immediate application to linear 
programming. In Chapter 1 we saw that the constraints on a linear 
programming problem can be written 


DOES 2), i=1,...,m. (2-91) 
j=1 
If we let a' = (ar, ..., an), then (2-91) becomes а'х|< = >)b, 
i= 1,...,m. Each one of the constraints requires that the allowable x 


be in some given closed half-space in E^ or, if the strict equality holds, 
lie on some given hyperplane. In addition, there are the non-negativity 
restrictions ту > 0, j = L,...,r. If we write a"+ = ef, then 2; > 0 
becomes a"*7x = ех > 0. Each of the non-negativity restrictions also 
requires that the allowable x be in some closed half-space. The region 
of E" defined by z; > 0,j = 1,..., r, is called the non-negative orthant, 
of Е". Now we can see that any feasible solution x must simultaneously 
be an element of each of the sets 


Xi = (xa x(« = 2), G=1,...,m+r, (2-92) 


where b; = 0, i = m-+1,...,m-+r. Thus the set of feasible solutions 
is the intersection of the m + r sets X; Therefore, the set of feasible 
solutions to a linear programming problem (if a feasible solution exists) is 
a closed convex set. Furthermore, this set is bounded from below (by 0) since 
E2 

The preceding analysis also shows that the set of solutions to a system 
of m linear equations in n unknowns, Ax = b, is a closed convex set. 
The set of solutions is the intersection of m hyperplanes. It is true, in 
addition, that the set of non-negative solutions to Ax — b, that is, the 
set of solutions with x > 0, is a closed convex set. Later we shall see that 
it is advantageous to convert any inequalities in the set of constraints to 
equations by introducing additional variables. We might note here that 
this conversion does not change the fact that the set of feasible solutions 
is a convex set. 
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We have already defined a convex combination of two points. This 
definition can easily be generalized to the concept of a convex combina- 
tion of m points. A convex combination of a finite number of points 
Xj, ..., Xm 18 defined as a point 


m m 
xe» ax, ш» |1-4d..m;! DD ws = 10 (2-3 
iml isl 
It is very easy to prove directly from the definition of a convex set that 
the set of all convex combinations of a finite number of points xi, . . . , Xm 
is a convex set. 


Convex POLYHEDRON: The set of all convex combinations of a finite 
number of points is called the convex polyhedron spanned by these points. 


The convex polyhedron spanned by n + 1 points in E" which do not 
lie on a hyperplane is called a simplex. In E?, a triangle and its interior 
form a simplex. The three points which generate the simplex are the 
vertices of the triangle. 


2-21 Convex sets and hyperplanes. In this section we state four 
important theorems which are useful in linear programming, decision 
theory, and game theory. 

THEOREM I: Given any closed conver set X, a point y either belongs 

to the set X or there exists a hyperplane which contains y such that all 

of X is contained in one open half-space produced by that hyperplane. 


This theorem is proved in {LA 6-6]. It is geometrically obvious in E? 
and E?, as Fig. 2-8 illustrates. If y is not in the set X, we can pass a 
line through y such that all of X lies in one open half-space produced by 
the line. 


T) 


cx =z 
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Given a boundary point w of a convex sel X; then cx = г is called a 
supporting hyperplane at w if cw = г and if all of X lies in one closed half- _ 
space produced by the hyperplane, that is, cu > z for all u € X or cu < z 
for all u € X. 


THEOREM II: If w is a boundary point of a closed convex set, then there 
їз at least one supporting hyperplane at w. This theorem is proved in 
{LA 6-6}. There may be more than a single supporting hyperplane 
at w. Consider, for example, Fig. 2-9. $ 


We have already shown that the set of feasible solutions to a linear 
programming problem is a closed convex set, and that the function to 
be optimized is a hyperplane. This hyperplane is moved parallel to itself 
over the convex set of feasible solutions until z is made as large as possible 
(if z is being maximized) while still having at least one point x on the hyper- 
plane in the convex set of feasible solutions. 

Hence, if à given hyperplane corresponds to the optimal value of z, 
then no point on the hyperplane can be an interior point of the convex j 
set of feasible solutions (recall that unbounded solutions are not called 
optimal). To see this, let us suppose that z = cx is an optimal hyper- 
plane, and that one of its points хо is an interior point of the set. We 
select an € > 0 such that every point in this e-neighborhood of xo is in X. 
The point x, = хо + (€/2)(c’/|c|) is in X, and ex; = z + (€/2)|c| > z. 
This contradicts the fact that z is the maximum value of the objective 
function. Thus, if xo is an optimal solution to a linear programming prob- 
lem, it must be a boundary point of the convex set of feasible solutions, 
and if z = cxo, then cx = z is a supporting hyperplane to the convex 
set of feasible solutions at Xo. 


THEOREM III: A closed convex set which is bounded from below has 
extreme points in every supporting hyperplane. 
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This theorem is especially important in linear programming because 
it says that if there is an optimal solution, then at least one extreme 
point of the convex set of feasible solutions will be an optimal solution. 
The number of extreme points is finite (proof of this will be presented later). 
Hence, if we had the means of selecting the extreme points of the convex 
set of feasible solutions, only a finite number of points would have to 
be examined to find an optimal solution to the problem. This is the basis 
of the simplex method. We move from one extreme point to a new one 
until an optimal solution is found. 

We shall now prove Theorem III. The hyperplane cx = 2 will be 
assumed to be a supporting hyperplane at xo to the closed convex set X 
which is bounded from below. The intersection of X and 5 = (x|cx = z} 
will be denoted by T. The intersection is not empty because xo € 7; 
furthermore, since X and 5 are closed convex sets, so is T'; 7 is also 
bounded from below since X is. 

Let us show that any extreme point of 7 is also an extreme point of X. 
If t is any point in T, and if t = Ах; + (I — X)xi, 0 < ^ < 1, where 
X; X2 € X, then xj, x € Т. This follows from the fact that ct = 
Мех + (1 — AX)ex, = z, and cx, > z, ех; > z because cx = z is a sup- 
porting hyperplane. Noting that À > 0, 1 — > 0, we see that ct = z 
requires that cx; = 2, cx, = z, that is, xy, xo € T. Thus, if t is an 
extreme point of T, there do not exist other points x1, x? in X such that t 
can be written as a convex combination of these points with 0 < А < I. 
Hence an extreme point of T is an extreme point of X. 

We still must prove that 7 actually has an extreme point; this will 
be done by constructing an extreme point. Out of all the points in 7’, 
choose the one with the smallest (algebraic) first component. "There is 
at least one such point since T' is closed and bounded from below. 

1f there are more than one point with a smallest first component, choose 
the point or points with the smallest first and second components. If a 
unique point is not obtained, choose the point with the smallest first, 
second, and third components, ete. Finally, a unique point will be ob- 
tained, since only one point can have all its components of minimum 
algebraic value. 

The unique point t* determined by the above process is an extreme 
point. If t* were not an extreme point, we could write 


t* = At, + (1 А O<A<1, tut ET, ti # t. (2-94) 


Suppose the unique t* was determined on minimizing the jth component. 
Denote the ith component of t*, tı, ta by (7, ta, tiz, respectively. Then 
from (2-94) and the definition of t*, it follows that (f = ta = lis, 
i= 1,...,j— 1. Then the jth component of (2-94) requires that 
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tf > min [tj 52] or tf = tji = tja; Either of these alternatives contra- 
dicts the fact that /f is the unique minimum for the jth component when 
the first j — 1 components are at their minimum values. Consequently. 
t* cannot be represented as a convex combination of any other two points 
in T (0 < ^ < 1). Hence t* is an extreme point, and the theorem is 
proved. 


Тнков®м IV: If a closed, strictly bounded conver set X has a finite 
number of extreme points, any point in the set can be written as a convex 
combination of the extreme points, that is, X is the set of all convex combi- 
nations of its extreme points. 


"This theorem is proved in (LA 6-8]. 


Ехлмрік: Suppose that we wish to write any point w inside a triangle. 
as а convex combination of the vertices (extreme points) X1, X5, Хз, that 
is, w = yxy ш > 0, Dui = 1. The situation is illustrated in Fig. 
2-10. First, draw a line from x2 through w. It will intersect the opposite 
side of the triangle at v. Then w = Axa + (1—39v,0 € M <1. 
However, v = Ах + (1 — Ag)xs. Thus 


W = Лх: + (1 — X)MX: + (1 — X)(1 — Ao) xg. 


Let ш = №(1 — X), ua = №, из = (1 — X) — №). Clearly, each 
ш 2 0, and ш + us + из = 1. The desired expression is W = Sux; 


From the definition of a convex polyhedron and from Theorem IV, it 
follows that every closed strictly bounded convex set with a finite number 
of extreme points is a convex polyhedron. Convex sets with a finite 
number of extreme points are essentially convex polyhedrons. However, 
they may have the form shown in Fig. 1-5, that is, they may not be 
strictly bounded, 


Figure 2-10 
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Ерве: Let xi, xa be distinct extreme points of the convex set X. The line 
joining them is called an edge of the convex set if it is the intersection of X 
with a supporting hyperplane. If x* is an extreme point, and if there exists 
another point x € X such that the line x* +- Xx — x*), all ^ > 0, is 
in X and is the intersection of X with a supporting hyperplane, then this 
line is said to be an edge of the set which originates at x* and extends to 
infinity. 


ADJACENT EXTREME POINTS: Two distinct extreme points ху, Xa of the 
convex set X are called adjacent if the line segment joining them is an 
edge of the convex set. 


2-22 Convex cones. A cone C is a set of points with the following prop- 
erty: If x is in the sel, во is ux for all и > 0. The cone generated by a set 
of points X = {x} is the set 


С = |yly = ux, all y > 0, and all x € X]. (2-95) 


In E?, E?, a cone as a set of points is often identical with the usual geo- 
metrical concept of a cone. Note that a cone is never a strictly bounded 
set (except in the trivial case where 0 is the only element in the cone). 
The point 0 is an element of any cone and is called the vertex of the cone. 
The negative C^ of a cone C = {u} їз the set of points C^ = |—u]. 
Naturally, C^ is a cone if C is. The sum of two cones Сү = {u}, C? = [v], 
written C, + С», ts the set of all points u + v, u € Ci, v € Сз. The sum 
Cı + О» ів а сопе. If C = {и} їз a cone, then C*, the cone polar to С, is 
the collection of points {v} such that v'u > 0 for each v in the set, and all 
u € C. It is easy to see that C* is а cone. Intuitively, a polar cone is the 
collection of all vectors which form a nonobtuse angle with all the vectors 
in C. Note that each v Є C* must form a nonobtuse angle with every 
vector in C. 

A cone is a convex cone tf it is a convex set, It can be proved [LA 6-9) 
that a set of points is a convex cone if and only if the sum v, -+ vo is in 
the set when vı, Va are, and if uv is in the set when v is for any и > 0, 
The sum of two convex cones is also convex. The cone generated by a 
convex set is à convex cone, The dimension of a convex cone C is defined 
to be the maximum number of linearly independent. vectors in C. 

Given a single point a # 0, a half-line, or ray, is defined as the set L = 
lyly = na, all > 0]. Note that a half-line is a convex сопе. A conver 
polyhedral cone C is the sum of a finite number of half-lines, i.e., C = Y Li. 
In this definition the term “sum” is used in the sense of sums of cones. 
If the point a; э 0 generates the half-line L;, then a convex polyhedral 
cone is the collection of points 
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The cone generated by a convex polyhedron is a convex polyhedral cone, 


ExaMPLE: Figure 2-11 shows the convex polyhedral cone generated 
by the half-lines 1, 2, 3. Note that any cross section of the polyhedral 
cone is a convex polyhedron. 


If A is ann X r matrix А = (a,,... , ar), then the set of points y = 
Ax = Doxyaj, all x > 0, is a convex polyhedral cone in E". The columns , 
aj of A generate the half-lines whose sum yields the polyhedral cone. 
Thus we see that there is a non-negative solution to the set of equations 
Ax — b if and only if b is an element of the convex polyhedral cone gen- 
erated by the columns of A. 

Any given finite number of points 81,..., & from E" can be thought 
of as generating a convex polyhedral cone C. If the maximum number of 
linearly independent points in the set 81,..., ar is n, then the set gen- 
erates а cone of dimension n. Suppose that we have in E" an n-dimensional 
cone generated by a,,...,8,. Out of the set а,...,а,„ let us choose 
&ny n — 1 linearly independent points by,..., Dai. These points 
determine a unique hyperplane cx — 0 through the origin in E^. It may 
or may not be true that all of the cone C will lie in one of the closed half- 
spaces cx > 0 or ex < 0. 

If C does lie in one closed half-space produced by ex = 0, we make 
the following definitions: The set of points Hp = {vlev > 0] is an extreme 
supporting half-space for the n-dimensional convex polyhedral cone С gener- 
ated by the points a,,..., a, if C lies in the half-space Hp and n — 1 
linearly independent points from the set ai, . . . ‚а, lie on the hyperplane 
су = 0. The hyperplane cv = 0 is then called an extreme supporting 
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hyperplane for the convex polyhedral cone C. It is important to note 
the difference between an extreme supporting hyperplane for a convex 
polyhedral cone and a supporting hyperplane for any convex set as de- 
fined earlier, An extreme supporting hyperplane must have n — 1 linearly 
independent points of the cone lying on it, while a supporting hyperplane 
need not have more than a single point in common with the set. 

It is clear that the intersection F of an extreme supporting hyperplane 
cv = 0 and the polyhedral cone С yields a collection of points which are 
on the boundary of C. Furthermore, F is itself a convex polyhedral сопе 
generated by the points from a,,...,a, which lie on the hyperplane 
cv = 0. We call F a subcone of C; F has dimension n — 1 since there 
are precisely п — 1 linearly independent points in F. In Fig. 2-11 any 
two of the vectors а, аз, аз uniquely determine a plane through the 
origin such that the cone € is contained in one half-space produced by 
the plane. Also, any two points from a;, a», as are linearly independent, 
and hence the resulting hyperplane is an extreme supporting hyperplane. 
The intersection of C with any one of the three extreme supporting hyper- 
planes yields a face of the cone C. This face is a cone—the cone F dis- 
cussed above. In general, we call the (n — 1)-dimensional convex poly- 
hedral cone F which is the intersection of an n-dimensional polyhedral cone C 
in Е" with an extreme supporting hyperplane a facet or face of the cone С. 

An n-dimensional convex polyhedral cone in E" generated by 4ı, ... , ar 
can have only à finite number of extreme supporting hyperplanes since 
there are at most r!/(n — 1)!(r — n + 1)! sets of n — 1 linearly inde- 
pendent points in the set a,,...,a,. Not every n-dimensional convex 
polyhedral cone in E" needs to have an extreme supporting hyperplane. 
The convex polyhedral cone may be all of E", and thus cannot lie in any 
half-space of E^. 

The following result is sometimes useful: If a convex polyhedral cone C 
in E" contains no vector x < 0, then С+ contains a vector y > 0, y #4 0. 
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PROBLEMS 
2-1. Find the products AB and BA (when they are defined): 


qae te д h B- 8 | > 
azı 22 boi b22 
1 


waf Fam в -|*l; 
STD 8 
9 


134 10 2 0 
(< A=]2 0 7]: B-[71 3]; 
5 6 9 4 5 6 


(d) A = [1,0,7,8], B = (2, 4, 9,0). 
2-2. Obtain the inverse of the following matrix: 
10 1 2 
A-|[39 7]- 
2.1. 6 


What is a product form of the inverse? 
2-3. Show that if A is the partitioned matrix 


rg s "i к i Qa nr - 
Azı А22 12 Abe 
2-4. Given a nonsingular matrix B = (bi, ..., ba). Show that B~'b,; = e; 


Also, show that Be; = b;. 
2-5. Show that 


In | =1. 
0^ "T, 
2-6. Using A`! = (1/|A|)A*, compute the inverse of the following matrices: 
4-1-2 
21 
(a) A = Н (b A =|0 1 0f- 
3 4 
8 4 5 
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2-7. Do the following vectors form a basis for ЕЗ? 
(а) a1 = [3, 0, 2], a2 = [7,0, 9), аз = [4, 1, 2]; 
(b) a1 = [1,1,0], аз = [3, 0, 1], аз = [5,2,1]; 
(с) a1 = [1,5, 7], a2 = [4, 0, 6], аз = [1, 0, 0). 


2-8. Show how the change of basis technique (discussed in Section 2~10) 
can be used to express a vector in terms of some arbitrary basis by starting 
with the unit vectors as a basis and then proceeding to insert the vectors of the 
arbitrary basis, How is this related to the product form of the inverse? Illustrate 
by expressing b = (6,4, 1] in terms of a; = [3, 1,2], аз = {1, 3, 9], and 
аз = [2,8,5]. 

2-9. Given the basis vectors eı = [1, 0, O], [0, 1,1), ез = [0, 0, 1] for З, 
Which vectors can be removed from the basis and be replaced by b = [4, 3, 6] 
while still maintaining a basis? Illustrate this geometrically. 

2-10. In the text it was shown under what circumstances a vector could 
be removed from a basis and replaced by another vector in such a way that the 
new set of vectors was also a basis, Suppose now that we wish to form a new 
basis by removing two vectors from a basis and replacing them by two other 
vectors. Try to derive the conditions under which the new set of vectors will 
be a basis, What are the difficulties? Is it easier to insert the two vectors one 
at a time or both together? 

2-11. Prove the Schwarz inequality |а'Ъ| < lal |b]. Hint: 


Da + bj? = А442 + аъ + [2 2 0 for all X. 
Thus the quadratic equation |да + b|? = 0 cannot have two different real 


roots. 
2-12, Find the rank of the following matrices: 


oats} oat nnd 


921 321 453 
(A-|ae2[|./0472|3 2 1p04-7|2 0 7) 
0 0 0 0 0 0 9158 


afte tC 
АЖЕ 
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Plot these vectors. Do all possible basic solutions exist for the following sets of 
equations? How many do exist? Are they nondegenerate? 


(az, аз, a4, ав)х = bi, (a2, a4, a5, a6)X = bo, 
(аг, a4, ав)х = bs, (a1, a2, a4, аз)х = bi. 


In Problems 2-14 through 2-16, find all existing basic solutions. Do all 
possible basic solutions actually exist? 


2-14. zı + 222 + 3х3 + 4z4 = 7, 
221 + 22 + 23 + 224 = 3. 
2-15. 821 + 622 + 1823 + z4 + 25 = 6, 
921 + z2 + 223 + 6z4 + 1025 = 11. 
2-16, 2x) + 312 + 423 + T4 = 2, 
zı + 22+ 723 + t4 = 6, 
8x1 + 2z2 + хз + 5z4 = 8. 
2-17. Which of the following sets are convex? 


(a) X = (а, 221821 + 223 < 6); 
(b X = (Iz zal > 220 < 3}; 
(с) X = {(zı, zallziza € 1,21 > 0, z2 > 0}; 
(d) X = (Ца, 2222 — 3 > —zi, 21 > 0,22 > 0). 
2-18. Sketch the convex polyhedra generated by the following sets of points: 
(а) (0,0), (1,0), (0,1), (1, 1); 
(b) (3, 4), (5, 6), (0,0), (2, 2), (1,0), (2, 5), (4,7); 
(e) (—1, 2), (3, —4), (4, 4), (0,0), (6, 5), (7, 1). 


2-19. Consider the triangle with vertices (0, 0), (2,0), (1, 1). Express the 
point (0.5, 0.5) as a convex combination of the extreme points. Do the same 
for point (0.3, 0.2). 

2-20. What difficulties would arise in the process of solving a linear program- 
ming problem if the constraints were open (i.e., have a > ог < sign) rather than 
closed half-spaces. Would the absolute maximum or minimum actually be 
taken on at a point in the set of feasible solutions? 


CHAPTER 3 
THEORY OF THE SIMPLEX METHOD 


“. , , il leads 
A long, sleep journey, through sunk 
gorges, o'er mountains in snow," 


Matthew Arnold. 


3-1 Restatement of the problem. The general linear programming 
problem was formulated in Chapter 1 as follows: We wish to find values 
for a set of т variables ту satisfying m linear inequalities or equalities (the 
constraints) of the form 


апт + anra Fg aqar lS = 20b. i-1..m (8-1) 


where one and only one of the signs <, =, > holds for each constraint, 
but the sign can vary from one constraint to another. Furthermore, the 
variables are to be non-negative, z j > 0,j = 1,...,г (the non-negativity 
restrictions), and are to maximize or minimize & linear form 


z = city + Cota b ott + oerte (3-2) 


All the aij, bj, су are assumed to be known constants. 

Recall that any set of x; which satisfies the set of constraints (3-1) is 
called a solution, Any solution which satisfies the non-negativity restric- 
tions is called a feasible solution. Any feasible solution which maximizes or 
minimizes the z of (3-2) is called an optimal feasible solution. We are 
trying to develop а procedure for finding an optimal feasible solution. 

In this chapter, the theoretical foundations of the simplex method will 
be considered in some detail. The simplex method is fundamentally a 
rather simple procedure for solving a linear programming problem. How- 
ever, some of the difficulties pointed out in Chapter 1 and the necessity 
of showing that the simplex method will lead to a solution in a finite num- 
ber of steps create certain problems whose elimination requires a con- 
siderable amount of diseussion. This discussion is apt to give the impres- 
sion that the simplex technique is much more difficult than it really is. 
Hence we suggest that the reader studying this chapter and Chapters 4, 5, 
6, and 7 attempt to keep in mind the basic simplicity of the simplex 
method. It should also be recognized, however, that a thorough compre- 
hension of the subject, which is necessary for an intelligent use of the 
technique, requires à discussion of the finer points. 

7 
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3-2 Slack and surplus variables. In general, it is much more con- 
venient to work with equations than with inequalities. For this reason, 
it is desirable to convert any inequalities in the constraints (3-1) into 
equations, so that a system of simultaneous linear equations is obtained. 
This conversion can be carried out very simply by introducing some 
additional variables, which are called slack and surplus variables. 

In the majority of the practical linear programming problems which will 
be formulated in this text, we shall find that the b; of (3-1) are all non- 
negative. From a strictly theoretical point of view, however, the b; can have 
any sign. Later we shall find it advantageous to have all b; > 0. Hence, 
at the outset, the necessary transformations will be made to ensure that 
each b; > 0. 

Suppose that in the original formulation of some constraint 1, b; < 0. 
Multiplying both sides of this constraint by —1, we obtain 


айй — dista < aqar |> = <) = (83) 


(Note that if the constraint has an inequality sign, the direction of the 
inequality is reversed after multiplying by (—1); that is, —5 < —1, and 
(—1)(—5) > (—1)(—1).) Multiplication by (—1) yields —b; > 0. 
Thus it is a simple matter to convert all the constraints into ones with 
b; 2 0. Hereafter, we shall assume that (3-1) represents a set of con- 
straints with each b; > 0. 

We would now like to convert the set of constraints (3-1) into a set of 
simultaneous linear equations. To simplify problems of notation, let us 
agree to number the constraints in the following order: (1) Constraints 
with < signs; (2) constraints with 2 signs; (3) constraints which were 
originally formulated as equations. Á 

Consider first the constraints having < signs: A typical constraint of 
this category (say constraint h) can be written 


Е 
У а; X b. (3-4) 


j=l 


Let us introduce a new variable Zr4n > 0, where 


E 
fep = Dy — У) а > 0. (3-5) 


j=l 


We call 2; a slack variable, because 0, can be considered to be the 
maximum amount available of resource h, and for any set of Zi Lanza; is 
the amount actually used. The difference between the amount available 
and the amount actually used is the slack (such as slack machine time). 
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Equation (3-5) can then be written 


r 
У) aga = da (3-6) 
ici 

By introducing an additional variable, the slack variable, we have con- 

verted the inequality (3-4) into an equality, (3-6). Furthermore, we note 

that for any set of z; (not necessarily non-negative) satisfying the in- 
equality (3-4), z,.,4 must be non-negative. 

Next consider inequalities having a > sign: A typical inequality in this 
set (say constraint k) can be written 


Уау > br (3-7) 


jel 


Then define a new variable arye > 0 by 


Tryk = E аку; — bi. (3-8) 
jel 


The variable гү. is called a surplus variable, because Ь can be considered 
to be the minimum amount of a product to be manufactured (or of a 
resource to be used), and for any set of xj, Laxjxj is the actual amount: 
manufactured (or resource used). The difference between the quantity 
actually produced and the minimum to be produced is the surplus over 
and above the minimum. Equation (3-8) can be rearranged to read 


А 
D а) — Хек = by. (3-9) 
jml 

The inequality (3-7) has been converted into an equality by the introduc- 

tion of a surplus variable, For any set of x; satisfying the inequality (3-7), 

2,4) 18 non-negative, 

Thus, proceeding by steps, we have converted the original constraints 

(3-1) into a set of simultaneous linear equations of the form 


D aast try = by b= Myer (3-10a) 
jot 
Уау — tre = be, К=мш+1,...,һ, (3-10b) 
j=l 
У ая; = bp р= + 1,...,т, (3-100) 
j=l 


Constraints with a < sign will be converted to the form (3-10a), those 
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with a > sign to the form (3-10b), and those having an equality sign will 
remain unchanged. We then have m simultaneous equations in n un- 
knowns. The number of variables will be between r and r + m, depending 
upon the number of slack and surplus variables which must be introduced. 

Note that if the original set of constraints (3-1) contained inequalities 
with € and > signs, multiplication by —1 would convert all inequalities 
with > signs to inequalities with < signs, with the result that only slack 
variables would be introduced. Similarly, every inequality could be con- 
verted to one having a > sign, and then only surplus variables would be 
introduced. However, it is not possible to carry out either one of the above 
transformations and still maintain all b; > 0. In the general case, when 
allb; 7 0, the introduction of both slack and surplus variables is necessary. 

The set of constraint equations (3-10) can be written in matrix form as 


Ax = туа +--+ + tnan = b, (3-11) 


where the а; are the columns of A. A slack or surplus variable Жы appears 
only in equation Т, where it was introduced to remove an inequality. Thus 
the column of A corresponding to ту is e; if z-4; is a slack variable, and 
—e; if z,,; is a surplus variable. The vector e; is, of course, the ith m- 
component unit vector. 

The matrix A can then be written (recall that by our convention, equa- 
tions containing slack variables come first) 


211 а12 а, 1 0 0 0 
021 a22 аз, 0 1 9:2. 10 
A apa 612 +++ 04.34. 0 0 "mi 0 (3-12) 
ар а» tU" ys 0 0 0 —1 
Qn аә ttt Amr 0 0 D 0 0 


ExaMPLE: Convert 
221 + 3x2 < 6, 


ү + 725 > 4, 
+ 22 = 3 


into а set of equalities. Introducing а slack variable хз > 0 into the first 
constraint, we obtain 


2z, + 325 23 = 6. 
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A surplus variable z4 > 0 is introduced into the second constraint to yield 
21 + Tra — 24 = 4. 


The third constraint is already in equation form. The resulting set of 
linear equations can then be written in matrix form: 


ie: aes wh ds л 6 
1 7 0 —1|}**}=)4]- 
rip ul ett Lid Mla e 


Thus far we have proceeded to convert the inequalities in the constraints 
into equalities by introduction of slack and surplus variables, without 
mentioning the effects of this conversion on the function to be maximized 
or minimized. Let us now consider the function to be optimized. Orig- 
inally, we had 


= 22 estj. (3-13) 
jml 


The number су is often called the “price” associated with the variable ту. 
(Of course, it is not a price in every case.) As yet we have said nothing 
about the price to be associated with a slack or surplus variable. Suppose 
that we assign a price of zero to every slack and surplus variable. "Then 
the conversion of the constraints to a system of simultaneous linear equa- 
tions does not change the function to be optimized: 


А 
г = ах ess F ertr F Orry F °° f Orr = oD Суй. (3-14) 
j=l 
We claim that optimizing (3-14) subject to the constraints expressed by 
the set of simultaneous linear equations (3-11) witha; > 0 (j = 1,...,”) 
is completely equivalent to optimizing (3-2) subject to (3-1) with z; > 0 
(j = 1,.. =r). To prove this, note first that if we have any feasible solu- 
tion to the original constraints (3-1), then our method of introducing slack 
and surplus variables will yield a set of non-negative slack and/or surplus 
variables such. that, (3-11) is satisfied, with all variables non-negative. 
Conversely, if we have a feasible solution x to (3-11), then the first л com- 
ponents of x yield a feasible solution to (3-1). Thus, there is a one-to-one 
correspondence between the feasible solutions to the original set of con- 
straints (3-1) and the feasible solutions to the set of simultaneous linear 
equations (3-11). 
Next it will be observed that if x* = [z],..., z5] > 0 is a feasible 
solution to (3-11) which optimizes (3-14), then the first 7 components of 
x*, that is, zT, . . . , ef, are an optimal solution to (3-1) and (3-2), with the 
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same optimum z*. To see this let us suppose that for Eqs. (3-1) and (3-2) 
a different solution, say 21, ..., #,, gives an improved value of z, say 2, 
over z1, ..., ту; then we need to note only that there are non-negative 
slack and/or surplus variables 2,41, ..., ĉn such that 21, ..., Ên isa 
feasible solution to (3-11), with 2 as the value of z. This contradicts the 
fact that z* was the optimal z for (3-11) and (3-14). Similarly, by annex- 
ing the slack and surplus variables to any optimal solution to (3-1) and 
(3-2), we obtain an optimal solution to (3-11) and (3-14). Thus we con- 
clude that if slack and surplus variables having a zero price are introduced 
to convert the original set of constraints into a set of simultaneous linear 
equations, the resulting problem has the same set of optimal solutions as 
the original one. 

‘The preceding discussion has shown that the general linear programming 
problem can be reformulated as follows: Find a vector 


X pn. dos dal 210 
satisfying a set of m simultaneous linear equations 
Ax = та +++: + дап = Б, (3-15) 
and maximizing or minimizing the objective function 
e = citit +++ + Cnty = cx. (3-16) 


‘The vector b is often referred to as the requirements vector, and the a; are 
called activity vectors. As noted previously, the c; in the objective func- 
tion are called the prices associated with the variables ту. 

The above formulation of a linear programming problem will be used in 
discussing the theory of linear programming. 


3-3 Preliminary remarks on the theory of the simplex method. In 
Chapter 1, we discussed the geometrical interpretation of linear program- 
ming problems in two and three dimensions. We observed that the set of 
feasible solutions to a linear programming problem described mathe- 
matically by (3-1) and (3-2) formed a convex region or convex set (if 
there was a feasible solution). Furthermore, when an optimal solution 
existed (recall that unbounded solutions are not called optimal), at least 
one corner, or extreme point, of the convex region was optimal. In Chapter 
2, we showed that, in general, the set of feasible solutions to (3-1) or (3-15) 
is a convex set, and if an optimal solution exists, at least one of the extreme 
points is optimal. For the examples in two and three dimensions, there 
was only a finite number of extreme points. We suggested in Chapter 2 
‘that, in spaces of higher dimensions, the number of extreme points is also 
finite (we did not prove this; the proof will follow from the result of the 
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present chapter). Hence, if we had a way of finding analytically the 
extreme points of the convex set of feasible solutions, only a finite number 
of solutions would need to be examined to find an optimal solution (if an 
optimal solution exists). We shall prove below the interesting result that 
the extreme points of the convex set of feasible solutions ate basic feasible 
solutions to (3-15). (Review the discussion of basic solutions in Chapter 2.) 
Thus an optimal solution to a linear programming problem will be соп- 
tained in the set of basic feasible solutions to (3-15), It was shown in 
Chapter 2 that there is only a finite number of basic solutions (and this in 
turn shows that there is only a finite number of extreme points). 

To determine an optimal basic feasible solution, we might attempt to 

` find all basic solutions (including those which are not feasible) and then 
select one which yields the optimal value of the objective function. 
There is only a finite number of basic solutions, and hence, in theory, this 
could be done. However, this procedure is extremely inefficient, because 
the number of basic solutions increases very rapidly as the number of 
variables is increased. Even a very modest problem involving four con- 
straints and eight variables could require the solution of 81/4141 = 70 sets 
of four simultaneous equations in four unknowns. In addition, an examina- 
tion of the basic solutions alone would never determine whether the problem 
had an unbounded solution. 

Naturally, we would like to find an explicit expression for an optimal 
solution which would allow the numerical values to be computed directly 
(or point out that the solution was unbounded), without using an iterative 
procedure. As indicated in Chapter 1, no such expression has been de- 
veloped. Since the more efficient techniques used in solving sets of simul- 
taneous linear equations are iterative procedures, it is not surprising that 
iterative techniques have been found to be the most efficient means for 
solving linear programming problems. 

Thus our present efforts have to be concentrated on developing an 
iterative method of solution which lies between the two extremes outlined 
above, i.e., between the extremes of examining all basic solutions and of 
obtaining an explicit expression for an optimal solution. The simplex 
method referred to in the first chapter is such a method; it proceeds in 
systematic steps from an initial basic feasible solution to other basic 
feasible solutions and finally, in a finite number of steps, to an optimal 
basic feasible solution, in such a way that the value of the objective 
function at each step (iteration) is better (or at least not worse) than at 
the preceding step. Because z is improved (or at least not worsened) at 
each step, the number of basic feasible solutions that must be examined 
before an optimal solution is found is usually much smaller than the total 
number of existing basie solutions. The simplex method also indicates 
whether there is an unbounded solution. 
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Instead of beginning with the proof that an extreme point of the convex 
set of feasible solutions is a basic feasible solution, we shall find it of 
advantage to approach the general linear programming problem from the 
strictly algebraic point of view. The reason for this lies in the fact that 
the general linear programming problem must be solved by algebraie 


methods. Although the geometric interpretation by means of convex sets _ 


contributes considerably to an intuitive understanding of linear program- 


ming, it is not possible to solve geometrically any but the most trivial | 


linear programming problems. In developing the algebraic theory, we 


automatically formulate the basic features of the simplex method. In ~ 
addition, the algebraic approach, which is completely independent of - 
geometrical considerations, nevertheless provides deeper insight into the | 
geometrical results. After the algebraic treatment of the theory, we shall 


return to the geometric interpretation and. prove that an extreme point 


of the convex set of feasible solutions is a basic feasible solution. Since | 


the algebraic approach is independént of geometrical concepts, the reader 


need not be familiar with the material on convex sets in order to under- . 


stand it. 

"The algebraie theory that we are about to consider was first developed 
by George Dantzig (published in Chapter XXI of Koopman's Activity 
Analysis of Production and Allocation [5]). Our assumptions and our order 


of development will differ somewhat from Dantzig's presentation. How- | 


ever, the final results will be essentially the same. 


First, we wish to comment briefly on some of the assumptions that will | 


be made during our algebraie development: We shall assume that for the 
set of constraints (3-15), 


r(A) = r(A), A, = (A, b); (3-17) 


i.e., the rank of the matrix of the coefficients is equal to the rank of the 
augmented matrix. If this is not true, the constraint equations are incon- 
sistent, and there is no solution to the linear programming problem. In 
addition, it will be assumed that 


r(A) = m, (3-18) 


where m is the number of constraint equations. This restriction implies 
that there are at least as many variables as equations.. If, in the original 
formulation, there were more equations than variables, then some of the 
equations would be redundant and could be omitted. If the number of 
variables is the same as the number of equations, (3-18) requires that 
there be a unique solution. If this unique solution is feasible, it is the 
optimal solution, and if it is not feasible, then no solution exists. However, 
in practice, we are not interested in cases where the problem admits of 
only one solution, for then linear programming is not needed. In the 
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typical case, there are more variables than equations. Then (3-18) ensures 
that none of the equations is redundant. When a practical problem is 
properly formulated, (3-18) is usually expected to hold; in other words, 
we expect that the constraints will be consistent, and that each constraint 
will add some new restriction to the problem. 

During the actual process of formulating a linear programming problem 
with a considerable number of variables and constraints, it becomes 
extremely difficult to be sure that the constraints are indeed independent 
(or even consistent in some cases). It would be most unfortunate if, 
before applying the simplex method, we had to determine whether (3-18) 
held. It can be difficult to compute the rank of a large matrix. Hence we 
hope that the simplex method will tell us whether (1) there is a feasible 
solution and (2) any redundancy exists in the constraints. This can indeed 
be done. In fact, although the original set of constraints may not satisfy 
(3-18), we shall always begin the simplex method with a set of constraints 
which does satisfy (3-18). This set may not be the set of constraints (3-15), 
but a set of constraints having the same number of equations and some 
additional variables, These additional variables are introduced into the 
original constraints (3-15) in such a way that (3-18) holds. By applying 
the simplex method to the augmented set of constraints we finally de- 
termine whether the original set of constraint equations possesses a feasible 
solution, and whether there is any redundancy. 

To sum up the above discussion, we shall develop the algebraic founda- 
tions of the simplex method, assuming that (3-18) holds for the constraints 
(3-15). This is what we expect to happen in the typical case. If we cannot 
determine immediately whether (3-18) holds, then we apply the simplex 
procedure to a new set of constraints for which (3-18) does hold. Ulti- 
mately, we find whether the original set of constraints has a feasible solu- 
tion and whether any constraint is redundant. In addition, we actually 
find a feasible solution if there is one, and determine which constraints are 
redundant. We can drop the redundant constraints and then apply the 
algebraic theory to the new set of constraints, Equation (3-18) will hold for 
this new set. Consequently, the restriction (3-18) is really no restriction 
at all. It is introduced only to simplify the theoretical discussion. 

Note: There are several aspects of the condition (3- 18) that merit further 
comment. First of all, the fact that r(A) — m does not imply that a 
feasible solution to Ax = b exists. However, as indicated above, the sim- 
plex method will always tell us whether there is a feasible-solution to the 
set of constraint equations. Furthermore, even when (3-18) holds and the 
set of constraint equations is consistent, we cannot conclude that the orig- 
inal set of constraint inequalities was consistent. Consider 


2z; + 3z» < 6, 21 + 312 2 7. 
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These two inequalities are clearly inconsistent. When slack and surplus 
variables are added, we obtain 


2x, + 3x2 + x3 = 6, 
22; + 34g — % = 7. 


This set of equations is consistent. One solution is 
хз = 6, zi = —7, 2 qp = 0. 


The difficulty arises because the slack and surplus variables are not all 
non-negative. It will be remembered that any solution to the set of con- 
straint equations (3-15) is a solution to the original constraint inequalities 
(3-1) if and only if all slack and surplus variables are non-negative. If 
there is no solution to (3-15) having all slack and surplus variables non- 
negative, then the set of constraints (3-1) is inconsistent. 


3-4 Reduction of any feasible solution to a basic feasible solution. In 
this section, we wish to show that if there is a feasible solution to the 
constraint equations (3-15), then there is a basic feasible solution. Recall 
that if Ax = b has a solution and r(A) = m, then b can be written as a 
linear combination of m linearly independent columns of A; hence there is 
a solution with no more than m variables different from zero. Now we 
wish to extend this result and'demonstrate that if there is a feasible solu- 
tion to (3-15), then there exists a feasible solution with at most m variables 
different from zero, which is furthermore a basic solution. In addition, 
we shall actually show how to construct a basic feasible solution from any 
given feasible solution. 

Later, for reasons already outlined, we shall restrict our attention to 
basic feasible solutions to linear programming problems. The results of 
this section ensure that if a feasible solution exists, then there is at least 
one basic feasible solution, and hence there do not exist cases where a linear 
programming problem has a feasible solution but no basic feasible solution. 
We shall now formally restate and prove this result. 

Given a set of m simultaneous linear equations in n unknowns (n > m), 
Ax = b, with r(A) = m. Then if there is a feasible solution x > 0, there 
is a basic feasible solution.* To prove this, assume that there exists a feasible 
solution with p < n positive variables. Number the variables so that the 


* We shall always think of a set of equations Ax = b as a set of constraints 
for some linear programming problem. Hence, we shall call any solution which 
satisfies x > O a feasible solution. Outside of the linear programming context, 
а solution x > 0 would be called a non-negative solution to the set of equations. 


| 
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first р variables are positive. Then the feasible solution can be written 


Y. зај =, (3-19) 


ji 
and hence 


>20 (= 1,...,р); zj—-0 G=pti,...,n). (3-20) 


Now the vectors a; associated with the positive variables may be linearly 
independent or dependent. 

Consider first the case where the set of a; (j = 1,..., р) is linearly 
independent. Then p < m. If p = m, the given solution is automatically 
a nondegenerate basic feasible solution. Since r(A) = m, we know from 
Chapter 2 that if p < m, then there are m — p columns of A which, 
together with the p columns, form a basis for E" and yield a nonsingular 
matrix. Thus we can form a degenerate basic feasible solution with m — p 
of the basie variables equal to zero. 

Tt remains to examine the ease where the vectors a; ( = 1,..., p) are 
linearly dependent. We shall show that under these circumstances it is 
possible to reduce the number of positive variables step by step until the 
columns associated with the positive variables are linearly independent. 

When the a; (j = 1,..., р) are linearly dependent, there exist a; not 
all zero such that 


Ў an, e 0. (8-21) 
jmi 
Using (3-21), we proceed to reduce some z, in 
Yz«-5 20 (уш, Тр (2) 


j=l 


to zero. The type of analysis employed here will appear a number of times 
in our discussion of linear programming, and the reader should make a 
special effort to understand it. 

Suppose that any vector a, of the p vectors іп (3-21) for which a, »* 0 
is expressed in terms of the remaining p — 1 vectors. Thus 


= – У) a ay. (3-23) 
jwr "T 
Substituting (3-23) into (3-22), we obtain 
4$ APR Y 
X (z РА = а; = b. (3-24) 
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Here we have a solution with not more than p — 1 nonzero variables, 
However, we are not sure that all these variables are non-negative. In 
general, if we choose a, arbitrarily, some variables may be negative. It 18 
interesting to note that if a, is properly selected, the p — 1 variables in 
(3-24) will be non-negative. 

To see how to choose а,, observe that we wish to obtain 


a ж > 0 (= 1,...,р), jr. (3-25) 


For any j for which a; = 0, (3-25) will be satisfied automatically. Whi 
ај 7“ 0, division by a; must yield , 


NT (PNE асир, (3-202) 
ej ar 
EL а е (3-26b) 
aj ar 


The inequality sign in (3-26b) is reversed because we have divided by & 
negative number. Equation (3-26) can be interpreted to mean that 
z,/a, must not be greater than any x;/a; which is >0, and not less than 
any zj/aj which is <0. This requirement suggests a way of choosing the | 
vector a, such that the p — 1 variables in (3-24) will be non-negative. 
Suppose that we select the vector a,, using* y i 

2 = min (22, a; > б}. (3-27) 
Then each variable in (3-24) will be non-negative, and a feasible solution 
with no more than p — 1 nonzero variables has been found. 

Equation (3-27) implies that we first compute all z;/a; for which | 
a; > 0 and then select the smallest one. The variable corresponding to 
this smallest z;/a; is then driven to zero by the substitution (3-23). The 
aj are, of course, those given by (3-21). We have to consider at most А 
values of xj/a;. There must always be at least опе о; > 0 since none 
the а; = 0; if all a; < 0, we simply multiply (3-21) by (—1) and obtaiti 
new aj > 0. 

We have constructed a solution with not more than p — 1 positive 
variables; all other variables are zero. If the columns.associated with the 


* Instead of (3-27), the expression 


T. 
а= так{ < I 


could have been used to select a,. This choice will also satisfy (3-26). However, 
the vector chosen in this way will not be the same as that obtained from (3-27). 
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positive variables are linearly independent, we have reduced the problem 
to one which hus already been examined, and we know that a basic feasible 
solution exists. If the columns associated with the positive variables are 
linearly dependent, we can repeat the same procedure and reduce one of 
the positive variables to zero. Ultimately, we shall arrive at a solution 
such that the columns corresponding to the positive variables are linearly 
independent (why?), and we know that there is a basic feasible solution, 

It is not necessarily true that the minimum in (3-27) will be unique. 
There may be two or more zj/a; with the same minimum value. In such a 
case, instead of reducing the number of nonzero variables by one, we drive 
to zero all variables xj for which z;/aj has the minimum value. Because 
of the possibility that the minimum is not unique, we had to mention 
in the preceding paragraph that after the first reduction not more than 
p — 1 variables will be different from zero. There may be less than p — 1 
positive variables. 


ExaMPLE: Consider the system of equations 
x18; + тга; + 2383 = b; 
a, = (2,3), a =([1,1), а= (4, 5, b= [1, 14). 


A feasible solution is 
2a, + Заз + аз = b. 


The rank of the matrix А = (81, 42, а3) is 2. Hence a basic feasible solu- 
tion exists with no more than two variables different from zero. Note that 


a; + aa — ag = O, 


во that in the notation of (3-21), a, = 1, аз = 2, аз = —1. To reduce 
the number of positive variables, the variable to be driven to zero is found, 
according to our above discussion, from 

REN 7 - min f2,3} = 2- 

ae min (4, a; > of min (7,3 3 
Thus we can eliminate the vector аз for which x2/a2 = 3/2 and obtain a 


new solution with not more than two non-negative variables. By (3-24) 
the values of the new variables 2; are 


3 1 
=y- ža;  n-:-3075: 2 =1 ج‎ ieg 


Only two variables are different from zero. Furthermore, а, аз аге 
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linearly independent. The basic feasible solution is 
Ãıaı + 2заз = b. 
This result can be checked since 
4 (2, 3) + $14, 5] = [11, 14]. 


If а; is eliminated instead of ag, then x, is driven to zero. However, _ 
the new solution is not feasible. This can be easily verified: For ал, 
#1ı/aı = 2, and the new values of the nonzero variables are 


ta = 3—2(2)=—-1; #,=1—2(—-1) — 3. 


This solution is basic but not feasible, which shows that a feasible solution 
cannot be maintained if an arbitrary a; (a; > 0) is eliminated. 

Now suppose that the criterion given in the footnote on page 82 is used _ 
to select the vector which is to be eliminated. Only one aj, namely аз, is 
negative; thus аз should be removed. For аз, z3/a3 = —1, and the new 
variables are 


&-72—(-D00-23 = 3 — DE) = 5; 


these yield indeed a basic feasible solution which is different from that 
obtained above. 


3-5 Some definitions and notations. Before proceeding with the theory 
we shall introduce some definitions and notations that will be used a great 
deal in future developments. The notation may at first seem unnecessarily 
complex; howeyer, its usefulness will become clear as the simplex method 
is developed. Since all quantities have one or two subscripts, it is important 
that the meaning of these subscripts be thoroughly understood. 

The constraints of a linear programming problem are now written as & 
set of m simultaneous linear equations in n unknowns, 


Ax — b. (3-28) 


The jth column of the m X n matrix A is denoted by &j(j = 1,...,1)) 
We shall deal almost exclusively with basic solutions to this set of con- 
straints. Let us form a matrix B whose columns are any m linearly inde- 
pendent columns from A. These columns of B form a basis for E", Matrix 
B will be used exclusively to indicate an m X m nonsingular matrix whose 
columns are m linearly independent columns from A. We shall sometimes 
refer to B as the basis matrix and to the columns of A in B as the columns 
of A in the basis. The columns of B will be denoted by bi, ..., bm. Itis 
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important to note here that b; may be any column of A, say a, 7; the same 
applies to the other columns of B. The numbering of the columns of B 
does not indicate which vector from A corresponds to that column. Hence 
we must know from other sources which columns of A correspond to the 
columns in B. 

Any column aj of A can be written as a linear combination of the columns 
of B. The following notation will be used to represent such a linear com- 
bination: 


aj; = бз P- F унь = У) vibi = Byz; (3-29) 
iml 


or 
y= Ba;  уу= ip Ym (3-30) 


A knowledge of the vector y; allows us to express ay as a linear combination 
of the columns of B. The ith component of y; is y;j. Note that the subscript 
j refers to the vector aj, and the subscript i refers to the column of B 
which ууу multiplies in (3-29). The vector y; will, of course, change if the 
columns of A which make up B are changed. We repeat that the subscript 
i does not indicate which vector from A is in column i of B. 

Any basis matrix B determines a basic solution to Ax == b. This basic 
solution, defined by an m-component vector хр, is 


хв = Bub, xp = [285 227, Fam): (3-31) 


It will always be implicitly understood that all п — m variables not 
associated with the columns of A appearing in B are zero. The subscript ї 
means that z ;۾‎ corresponds to column b, of B; however, it does not indicate 
which variable x; of (3-28) zp; is. If ату із in column 2 of B, then тва == 
түт. By knowing which vectors from A are the columns of B we are able 
to determine the variables of x referred to by the zp;. If zn; > 0, we shall 
say that b; is in the basis at a positive level, and if za; = 0, b; is in the 
basis at а zero level. Recall from Chapter 2 that the variables rj; are 
called the basic variables, and the remaining n — m variables are called 
the nonbasie variables, Also recall that a basic solution is said to be 
degenerate if one or more of the basic variables vanish. The variables “in 
the basic solution” are the basie variables. 

Corresponding to any хр; we define ап m-component row vector св 
containing the prices of the basic variables as: 


Св = (CBI. <. + Cam): (3-32) 


The components of cp are cp; and the subscript ¢ means that cp, is the 
price of variable zg;. If ауу is in column 2 of B, then caz = су. 
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For any basie feasible solution, the value of the objective function z is 
given by 
2 = cíp, (3-38) 


since all nonbasie variables vanish. 


Next we shall consider an equation similar in structure to (3-29). A 
new variable 2; is defined by 


m 
2; = усві + +++ + YmiCam = У) Yaoi = сву;. (3-34) 
ii 


For each a; in A, there is a 2;; the z; corresponding to a; changes as the 
columns of A in B are changed. Later an economic interpretation will be 
given to z;. 


ExaMPLE: The above definitions and notation will be applied to the 
following linear programming problem: 
5 
35za;— b, alla; > 0; (3-35) 
j=l 
a = [4,1], а = [2,2], as = [1,3], а; = [1,0], 
as = [0, —1]), b= [2,1]; 
max z = 2z; + x2 + 323. 


The variables x4, 25 can be considered to be slack and surplus variables, 
respectively. The prices corresponding to these variables are zero. 

The vectors а, аз are linearly independent and form a basis for E". 
They can be used to form a basic solution. A basis matrix B will be formed 
by inserting аз into the first column of B, and a; into the second column 
of B, so that 


В = (bi, b2) = [ 1 


and bı = аз, bo = а. The basic solution with all variables except 21, . 
хз equal to zero is feasible, and is given by 


sh - 0-08 


with 22 = z4 = zs = 0. The variables in the basis are zp = 23, 
752 = zı. The prices corresponding to these variables are cg; = сз = 3, 
сво = сү = 2. Thus for this basic solution, 


св = (3,2). 
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Any other vector a; in A can be written as a linear combination of the 
basis vectors, This requires computation of the y; for aj. Consider ay; 
we obtain from (3-30) 


нв - iL 1 iB] = Af] - nz) 


az = узб + yoobs = 491283 + 42281. 


The value of 2з [defined in general by (3-34)] corresponding to аз is 


given by 
ó 4\ _ 20 
а = ез = 3(8) +2(4) n 


For this basic feasible solution, the value of z is given by 


2 5X. 16 
fx atem з(2) +2(5) п 


3-6 Improving a basic feasible solution. Suppose that а basic feasible 
solution to the linear programming problem (3-15), (3-16) is given by 
хв = B™'b. The value of the objective function for this solution is 
2 = свхв. In addition, we assume that for this basic solution the y; = 
В-!а; and 2; = сву; are known for every column ау of A not in B. 

We now wish to examine the possibility of finding another basic feasible 
solution with an improved value of г. We shall confine our attention to 
those basic feasible solutions in which only one column of B is changed; 
ie, we remove one column of B and replace it by some other vector 
from A. In Chapter 2, we discussed the method of changing a single vector 
in а basis. For this case, it is very easy to determine whether the new set 
of vectors forms a basis. We saw that if 


aj = Ууу, 
ii 


then а; can replace any vector b, for which y,; ¥ 0, and the new set of 
vectors will still form a basis. Let us then select any a; not in B for 
which at least one y;; # 0 and insert it into the basis matrix B. If we 
decide to remove column b, of B (y+; # 0) and replace it by aj, it follows 
that 


bm ay 545, (3-36) 
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The original basic feasible solution can be written 


x їв; = b. (3-37) 


fel 


Eliminating b, by (3-36), we obtain for the new basic solution 


m m 

рУ (es: — zar vd) bi + aj = b. (3-38) 
r= Yri, Yri | 
ter 


The new basic solution must also be feasible. This requires that 


zpi = тен > 0, 17 т, (3-39a) _ 
Tar сы, (3-39b) 


We cannot remove any b, with y,; z 0 and expect that (3-39) will 
hold. In fact, we immediately see from (3-39b) that if хр, ғ 0, we must 
have yr; > 0. Interestingly enough, we can now show that if a; has at 

` least one уу > 0, and if the proper column of B is removed and replaced 
by aj, then the new basic solution will be feasible. Note that if y,; > 0 
and yj < 0 (i = r), then (3-39a) is automatically satisfied for this i. 
Thus we only need to be concerned with the y;; > 0. We must select from 
the b; with yi; >0 the vector b, to be removed in such a way that (3-392). 
holds. ‘When yi; > 0, the requirement (3-39a) can be written 


SEE ERE „у, 
Yij Yri 


It is now clear that if we determine the column r of B to be replaced by 
means of 1 


Ве. min (23554; eT | 
Yri rs Vi ug o} 5 (3-40) 


then the new basic solution will be feasible. 
Let us summarize what we have said above and, at the same time, 
introduce some additional notation. We begin with a nonsingular matrix. 


В = (bi... , by), 


whose columns аге m columns from A, It is assumed that xg = B~'b isa 
basie feasible solution to the linear programming problem. We then 
select an arbitrary column a; from A which is not in B. If a; has one or 
more yi; > 0, y; = В-'!а;, we have shown that we can remove some 
column from B, replace it by ay, and thereby obtain a new basic feasible 
solution. It is not possible to select arbitrarily the column of B to be 
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removed if the new basic solution is to be feasible. However, if the selec- 
tion is done by means of Eq. (3-40), then the new basic solution will be 
feasible. Let us denote by B = (6,,..., bn) the new nonsingular matrix 
obtained from B by removing b, and replacing it by aj. The columns of the 
new matrix are given by 

$;-b, ir; Б, =a; (3-41) 


If the new basic feasible solution is denoted by $5, then £5 = B~'b; 
from (3-38) we obtain the values of the new basic variables in terms of 
the original ones and the y;;, that is, 


pg; = pi — am үй) i» r, (3-42a) 
7 
“250; 
tm =” (3-42Ь) 


At this point the reader should be careful to take note that we have 
obtained a new basic feasible solution to the set of constraints (3-15) by 
changing one of the variables in the basic solution. Consequently, the 
m — 1 variables that could be different from zero in the original basic 
solution can still differ from zero in the new basic solution, However, in 
general, the actual values of the variables will change, and their new 
values are given by (3-42a). On the other hand, the new variable, fpr, 
corresponding to the rth position is different from that in the previous 
solution. For example, if аз were b, (thus implying x”, = тз) and ay 
replaced b,, that is, аз, then £p, = 27. A different variable appears in the 
rth position, while in all others there are the same variables (usually with 
different values). The original variable in the rth position is driven to zero. 

We shall now turn to some of the more subtle points of the above dis- 
cussion. Note, first of all, that the minimum in (3-40) may not be unique. 
There may be two or more different columns of B for which the same 
minimum value is obtained. In this case, we can remove any one of the 
columns for which the minimum is taken on, and the new basic solution is 
feasible. It is important to observe what happens to the new variables 
still in the basic solution corresponding to the columns for which the 
6 of (3-40) was the same as that for the removed column. From (3-42a), 
we immediately see that since хасу = Zar/Yrj, nı = 0 for those 
columns that have the same value of @ as the removed column. In other 
words, if the minimum in (3-40) is not unique, the new basic feasible 
solution will be degenerate. In the new basis, all new basic variables will 
vanish for those columns that have the same 6 in (3-40) as the eliminated 
column. 

Next let us study what happens if the 0 of (3-40) is zero. Examination 
of Eq. (3-40) demonstrates that @ can be zero only if rz, = 0, that is, if 
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we start out with a degenerate basic solution. In this case, (3-42) immedi- 
ately shows that 25; = zp; (i 7 т), û, = 0. The values of the variables 
common to the old and new solutions do not change, and the new variable 
is at а zero level. Hence the new basic feasible solution is also degenerate. 
It does not follow that the new basic feasible solution will always be de- 
generate if the original one was. If y;; < 0 for every xp; which is zero in 
the original basic solution, none of these variables enter into the compu- - 
tation of (3-40). -Then 0 will be positive, and the new solution will be | 
nondegenerate, although we started out with a degenerate solution, pro- 
vided that r is uniquely determined. 

We might go one step further and note that the only reason for 
our requirement, yry > 0 is to ensure that zg,/y;; > 0. However, if | 
Xpy = 0, then it is not necessary that y,; > 0. If in the initial basic 
solution there is some basic variable xg, = 0 and у; < 0, then we can _ 
remove column r of B and replace it by ау. No computation like (3-40) is 
needed; since xg, = 0, the new values of the variables will be the same as _ 
the old ones and hence will be non-negative. These finer points discussed 
in the last three paragraphs will be of some use in later developments. " 

Our main interest, was in finding а new basic feasible solution with an | 
improved value of the objective function. Having found a new basic feasi- _ 
ble solution, we must determine whether z is improved. The value of the — 
objective function for the original basic feasible solution is z = c5Xg = 
Xen; The new value of the objective function 2 is given by 2 = ёвЁв _ 
= Уёвёв:. Note that ép; = св; (i # r) and £p, = cj, since only the 
price of the new variable entering the basis changes. Therefore, using 
(3-42), we obtain 


t= > сві (es: Br 2) EPI e, (3-43) 
íci Yrj Уј 
ier 


In the summation it is desirable to include the i = r term which is 
све (en, — ZB, ш) = 0. 
Yri, 
Because the term is 0, we can add it without changing 2. Thus 


m 
iz X ea (s = m) "E Bre, 


i=l Yri. Yri 
or 


4= » BiBi — —— uy cna + 20). (3-44) 


iml i=l 
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Note, however, that Усу is exactly what we defined to be zj. The 
expression for 2 reduces to 
#= a TE (ej — z) = 2 + Ole; — 2) (3-45) 

Here we have obtained a very interesting result. The new value of the 
objective function is the original value plus the quantity 6(c; — z;). Now 
zj = ©вуу, and it is known for the original basic solution (note that 2; 
refers to the original basic solution, not the new one), and c; is the price 
associated with the newly introduced variable ту. Assume now that we 
wish to maximize г. If 6(c; — гу) > 0, then 2 > z, and we have obtained 
an increase іп z. Since @ > 0, we cannot have 2 > z unless cj — z; > 0 
(orz; — cj < 0). Hence a restriction is placed on the choice of the vector 
a; to be inserted into the new basis. Up to the present, this vector has been 
arbitrary. We have just shown that if we choose any vector a; for which 
cj = гу > 0 and at least one уу > 0, we can obtain a new basic solution 
with 2 > z. If our original basic solution was not degenerate, then 0 > 0, 
and we have a definite increase in z; that is, # > z. When the initial basic 
solution is degenerate and we choose a vector aj for which су — 2; > 0 
and at least one угу > 0, we then may or may not have an increase in 2, 
depending on whether 6 > 0. In any event, when c; — z; > 0, we are 
sure that 2 is not less than г. By inspection of (3-45), we see that if we 
insert the variable z; into the basic solution and drive the variable corre- 
sponding to the rth column of B to zero, we gain 6cj. However, the intro- 
duction of 2; causes the remaining variables to change their values, and 
this causes a change of —6z; in 2. Hence c; — z; may be thought of 
intuitively as the change in z per unit increase in the value of ту when 
all other nonbasie variables are maintained at a zero level and the basic 
variables are allowed to change, as they must in order to maintain a solution. 

If we are minimizing rather than maximizing z, it is obvious that aj 
must have су — 2; < 0, in order that 2 < z. 

We can summarize the above discussion as follows: 

Given а basic feasible solution xg = В 'Ь to the set of constraints Ax = b 
for a linear programming problem, with the value of the objective function for 
this solution being z = свхв. If for any column aj in A, but nol in B, the 
condition cj > гу or 2; — cj < 0 holds, and if at least one уу > 0, 
i=1,...,m, then it is possible to obtain a new basic feasible solution by 
replacing one of the columns in B by aj, and the new value of the objective func- 
lion 2 satisfies £ > z. Furthermore, if the given basic solution is not degen- 
erate, 2 > 2. Similarly, if for some aj in A, but not in B, cj < zj or z; — 
cj > 0 and at least one yij > 0, then it is possible to obtain a new basic 
feasible solution by replacing one of the columns in B by aj, and the new value 
of the objective function £ satisfies 2 < г. Also, if the given basic solution 
is nol degenerate, 2 < z. 
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Ехлмрі: Consider the following linear programming problem: 
хла + 2982 + 23a3 = b, 2; 20, j= 1, 2,3, 
max 2 = C21 + C272 + 6373; 


a —[53, a = [2,4 
a;= [2,1], ъ= (8,7, 


€i = 3, сз = 2, Сз = 1. 


Let us form the basis matrix B with ag in column 1 and a2 in column 2, i.e., 


The corresponding basic solution is feasible since 


verit 00-0 


For this basic feasible solution, 
<j 1 
у= В а= [i] св = (1,2), 


2= вур = 2, 2;—06—52—3- —1, 
1(3) + 2(1) = 5. 


According to the theory developed above, we should be able to replace one 
of the columns in B by a; and obtain a new basic feasible solution with, an 
improved value of the objective function, since z; — су < 0, уу > 0, 
i = 1, 2, and the given basic feasible solution is not degenerate. 

We determine the vector to be removed, using 


EBr in (22 (3. 221,1 tre 2в2 
— = піп (= 5 Оу = = = 4 = = y 
Yri i ier 0 ) xs (j 111/2 yan Chee 


її 


2 — CpXn 


The minimum is taken on for r = 2, and the second column of B, that is, 
az is removed. Thus 


=2 д) = 2+2=7>2=5. 
Уту 
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We can easily check this result by finding в, ёз and using 2 = 6757: 


ёв = (1,3), 2 = др. = 7. 


The value of 2 verifies that obtained above. We have found a new basic 
feasible solution with аз, a; іп the basis, and 2 > z. 


3-7 Unbounded solutions. In the foregoing discussion we assumed that 
if we inserted a; into the basis and removed some column to form a new 
basis, there was at least one yj; > 0,7 = 1,..., m. Let us now consider 
what happens if we insert an aj for which all yj; € 0, i = 1,...,m. It 
has been noted in the previous section that if a new basic solution is to 
be obtained on inserting such an ay, then aj must enter the basis at a 
negative or a zero level (since y,; will be negative), thus implying that 
the new basic solution will not be feasible unless aj enters at a zero level. 
Now, instead of trying to obtain a new basic solution, we shall study the 
solutions which can be formed by allowing m + 1 variables (х; and the 
хпв) to be different from zero. We have the basic feasible solution 


Y bs b, (8-46) 


isl 


with the value of the objective function being 2 = cxa. Suppose that 
we add and subtract 0а;, @ any scalar, in (3-46) to obtain 


У) тв; — Oa; + ба, = b; (3-47) 


tet 


but 
т 
— ba; = —0 У) иг. (3-48) 


iwl 


If (3-48) is substituted into (3-47), we obtain 


5 (zn; — буу); + ба; = b. (3-49) 
tml 


When 0 > 0, then zp; — Oyi; > 0 since, by assumption, yj; < 0, i = 
1,..., m; therefore (3-49) is a feasible solution in which m +- 1 variables 
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can be different from zero. However, it is not, in general, a basic solution. 
Note that (3-49) is feasible for any 6 > 0. 

Next we shall compute the value of 2 for the feasible (nonbasic) solu- 
tion (3-49): 


i= 2$ citi = У) сві(=ві — бузу) + c;0 = z + O(c; — г). (3-50) 


i=l 


Then by making 6 sufficiently large ê can be made arbitrarily large if 
cj — 2; > 0, and arbitrarily small if c; — 2; < 0. We can summarize 
the above development, as follows: 

Given any basic feasible solution to a linear programming problem. If 
for this solution there is some column a; not in the basis for which 


2 oj < 0 .and yf S 0 (—1,:..,m), (3-51) 


then there exist feasible solutions in which m + 1 variables can be different 
from zero, with the value of the objective function being arbitrarily large. In 
such a case, the problem has an unbounded solution if the objective function 
is to be maximized. Similarly, if for some aj, 


2; —6; 20 and SO (1—31,:..,m) (3-52) 


then there exist feasible solutions, in which m + 1 variables can be different 
from zero, with the value of the objective function being arbitrarily small. In 
such a case, the problem has an unbounded solution if z is to be minimized. 
The preceding statement is a rather pleasant surprise, because it gives 
us some information about the annoying problem of unbounded solutions. 
In fact, it will be seen that the above results enable us to determine always 
whether a linear programming problem has an unbounded solution. As 
indicated in Chapter 1, we do not expect real-world problems to possess 
unbounded solutions. In fact, linear programming is useful because there 
are limiting factors which prevent us from making the function to be 
optimized arbitrarily large. However, some mistake in the formulation of 
a linear programming problem may at times lead to an unbounded solution. 
For this reason, it is most desirable to have a computational technique 
which will show whether there is an unbounded solution. It would be 
dangerous if a technique selected only the basic feasible solution with the 
largest value of the objective function, for in such a case the problem 
might actually have an unbounded solution and we would not know it. 
Fortunately, with the help of the information developed above, the 
simplex method does indicate whether there is an unbounded solution. 
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ExaMPLE: If in the preceding example a, = [—3, —3], then 


yis Ba, = ii ui ped m Ej: 


.5— = 0 = ر2 


An unbounded solution exists. If бау, 0 > 0, is added to and subtracted 
from the basic feasible solution ag + Заз = b, we obtain 


ay + ( s 9) 22 + (8 + баз = b. 


This is feasible so long as 8 > 0, and 
2 = z+ O(c) z) = 2+ 50. 


If 0 is increased sufficiently, 2 can be made arbitrarily large. The feasible 
solution will have three variables different from 0. 


3-8 Optimality conditions. Let us consider a linear programming 
problem for which we wish to maximize the objective function. Sec- 
tion 3-6 has shown that if we start with a basic feasible solution and if 
there is a vector a; not in the basis with z; — су < 0 and at least one 
Yi; > 0, i = 1,...,m, then there exists another basic feasible solution 
with the new value 2 of the objective function at least as large as the old 
value, г. If degeneracy is not present, then 2 > 2. 

For the moment, we shall assume that we never encounter degeneracy. 
Under these circumstances, we can move from one basis to another, chang- 
ing one vector at a time, so long as there is some ау not in the basis with 
z; — cj < 0 and at least one уу > 0. At each step, z is increased, This 
process cannot continue indefinitely because there is only a finite number 
of bases, In the absence of degeneracy, no basis can ever be repeated 
because z increases at each step, and the same basis cannot yield two 
different values of z. However, the process can terminate in only one of 
two ways: 

(1) One or more 2; — су < 0, and for each z; — су < 0, уу € 0 for 

ай = 1,...,m. 
(2) All z; — су > 0 for the columns of A not in the basis. 


If the process terminates with condition (1), we know that there is an un- 
bounded solution. This leads us to suspect that if the process terminates 
with condition (2), we have an optimal basic solution. This is indeed 
the case, as we shall now show. 

Assume that we have a basic feasible solution xz = B~'b to a linear 
programming problem Ax = b, x > 0, max z = cx, and that the value 
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of the objective function for this solution is zo = ¢sxz- In addition, 
suppose that z; — cj > 0 for every column a; of A not in the basis matrix 
B. It is easy to demonstrate that, subject to the constraints Ax = b and 
the non-negativity restrictions x > 0, zp is the maximum value of the 
objective function z = cx. 

Let 1 
2£;20, 4—1,...,n " zj8;------ ona, = b, (3-53) 


be any feasible solution to Ax = b. The corresponding value of the 
objective function, denoted by z*, is } 


z* = ex +++ + сата: (3-54) 


Any vector a; in A can be written as a linear combination of the basis 

vectors in B: | 
т 

aj = P vibi. (3-55). 

tel 1 


On substituting (3-55) into (3-53), we obtain 


zj 9^ Mabi ++ e Y vbi cs В, 


ї=1 i=l 
or a 
n n ie 
[> ess} dosis cade рэ zm] b, — b. (3-56) 
j=l j=1 h 


Recall, however, that the expression of any vector in terms of the basis — 
vectors is unique. We must conclude that 


n 
tpi = yx THz, t= 1,...,m. (3-57) 

per 3 

Next we turn our attention to 2*. By hypothesis, z; > с; for every | 

column of A not їп the basis, For those columns of A that are in the basis, 


yj = B'a; = B^b; = e; (3-58) 


if aj is in column 4 of B. Hence гу = Cay; = свег = cp; = cj. Thus, 
for the columns of A in thé basis, it is always true that ap e p= 0. We 
conclude that for the case under consideration 2; > €; for every column 
іп A. Consequently, using 2; > c; in (3-54), we see that 


Amp deed зал Det (3-59) 
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since each ту > 0. Now we use the definition 


m 
am >D сн 


iml 


in (3-59) and obtain 


[È as |e meat [ 


j=l 


a 
br — сва > 2“. 


j=l 
However, because of (3-57), we have 
20 = zpiCmick ss Хвт©вт > #*. (3-60) 


This proves that zo is at least as large as the objective function for any 
other feasible solution, and therefore zo is the maximum value of the 
objective function. 

We can summarize the above results in the following statements: 

Given a basic feasible solution хв = В-'Ь with 20 = свхв to the linear 
programming problem Ax = b, x > 0, maxz = cx such that z; — c; > 0 
for every column ау in A. Then zo is the maximum value of z subject to the 
constraints, and the basic feasible solution is an optimal basic feasible solution. 

Given a basic feasible solution xg = B^ !b with zo = срхв to the linear 
programming problem Ax = b, x > 0, minz = cx such thal 2; — cj < 0 
for every column aj in A. Then zo is the minimum value of z subject to the 
constraints, and the basic feasible solution is an optimal basic feasible solution. 

Note that our proof did not depend on any requirement that the basic 
feasible solution xp be nondegenerate. If all z; — су > 0, then xg is an ` 
optimal solution regardless of whether or not it is degenerate. 

Any linear programming problem falls into one of three mutually 
exclusive and collectively exhaustive categories: (a) There is no feasible 
solution; (b) there is an optimal solution; (c) the objective function is 
unbounded. If there is a feasible solution and hence a basic feasible solu- 
tion, only two possibilities remain: There is either an optimal solution or 
an unbounded solution. If we are given a basic feasible solution which is 
not optimal, the above results have shown that, in the absence of degen- 
eracy, we can proceed step by step, changing a single vector in the basis 
at a time, and reach either condition (1) or condition (2) in a finite number 
of steps: Condition (1) indicates that there is an unbounded solution, and 
condition (2) that we have an optimal basic feasible solution. These are 
the only two alternatives, and hence all possibilities have been accounted 
for. 

Note that in the absence of degeneracy, we have proved that if there 
is an optimal solution, one of the basic feasible solutions will be optimal. 
This is a very important result. Stated in another way, it means that an 
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optimal solution to a linear programming problem never need have more 
than m nonzero variables. We have also shown that, in the absence of 
degeneracy, there will be an optimal basic feasible solution having all 
2j — 6 20. 


ЕхАмргЕ: Consider the basic feasible solution characterized by the 
basis B (В = [î jl, B = Q/3)L.i —3]) (see example, р. 92). The 
only vector not in the basis is аз. For this vector 


a A - 3b 


Z2 ¬ C2 = ёвуз — с; = 4—2 = 2 > 0. 


Recall that for vectors in the basis the 2; — c; = 0. It follows from our 
discussion that the basic feasible solution with 8;, аз in the basis is an 
optimal solution. Note that a; + (1/2)аз + 2a3 = b is a feasible non- 
basie solution. The value of the objective function for this solution is 


#= ex = 1(3) + $2) +211) = 6, 


and this is indeed less than the optimal value z = 7. 


We must now point out the difficulties that the problem of degeneracy 
introduces into the above arguments. First of all, when degeneracy is 
present, we are not sure that the insertion of a vector will produce a new 
value of z which will be greater than the previous one. The value of the 
objective function may remain unchanged, and for this reason, a basis 
may be repeated. This means that we are not sure that. condition (1) or 
(2) can be reached in a finite number of steps. We may possibly enter 
a loop and cycle, forever repeating the same Sequence of bases, without 
changing the value of 2; this would imply that an optimal solution (or an 
indication of an unbounded solution) would never be achieved. Hence, 
the foregoing argument does not ensure that if degeneracy appears, there 
is an optimal basic feasible solution with all 2j — cj 2 0. In fact, even if 
there is an optimal solution, we cannot be sure that one of the basic solu- 
tions will be optimal. 

If it could be demonstrated that a basis never needed to be repeated, 
then, since the number of bases is finite, the above process would terminate 
in one of two ways: (1) z; — cj > 0 for all j; or (2) for all meo, 
Yi; € Oforallz. Condition (1) would imply that an optimal basic solution 
had been found, and (2) would indicate the presence of an unbounded 
solution. "Therefore the resolution of the degeneracy problem centers 
about the task of proving that a basis never needs to be repeated. Once 
this is done, it follows that at least one of the basic feasible solutions will be 
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optimal, and at least one of the optimal basie solutions will have all 
z; — c; > 0, Later we shall resolve the degeneracy problem by showing 
that a basis never needs to be repeated. 

For the present time, we shall assume that even when degeneracy 
appears, one of the basie feasible solutions will be optimal, and that one 
of the optimal solutions will have all z; — c; > 0. Incidentally, when 
degeneracy is present, there can be optimal basic solutions with one or 
more 2; — су < 0. An example of this will be presented later. It also is 
of interest to note that one of the basic feasible solutions can be shown to be 
optimal in a way that is completely independent of any degeneracy argu- 
ments. This will be done when the correspondence between basic feasible 
solutions and extreme points is discussed, since we know from Chapter 2 
that if there is an optimal solution, at least one of the extreme points of 
the convex set of feasible solutions is optimal. 


3-9 Alternative optima. The optimal value of the objective function 
for any linear programming problem is unique. However, the set of 
variables which yield the optimal value of the objective function does not 
need to be unique.’ Such a case was illustrated in Chapter 1, Fig. 1-2. We 
saw that if the line representing the optimal value of z lay on one side of 
the convex set and if two extreme points yielded optimal solutions, then 
any convex combination of the extreme points was also an optimal solu- 
tion. Now we wish to develop the generalization of this case. 

Suppose that a given linear programming problem has k different basic 
feasible solutions, X1, . . . , Xs, Which are optimal. We assume that each x; is 
an n-component vector and therefore contains the values of all variables 
(not only the basic ones). Consider any convex combination of these 
solutions: 


k 
х= um, me 20 CHL, Уы. (81) 


i=l 


Since each x; > 0, and ш > 0, it follows that x > 0. Furthermore 
Ax; = b, and hence Ax = b. Consequently, x is a feasible (but perhaps 
nonbasic) solution. If 20 = maxz = сх; (i = 1,..., k), the value of 
the objective function for x is 


k k 
cx = У) ر2 = عم‎ ngo = 2o. (3-62) 


iml iml 


Hence x is an optimal solution. We have proved that if X1, ..., Xy are k 
different optimal basic. feasible solutions to a linear programming problem, 
then any convex combination of Xi, . . - , Хь is also an optimal solution. 
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"This shows that if there are two or more different optimal basic feasible 
solutions, there will be an infinite number of optimal solutions. However, 
in this case, not all optimal solutions will be basic, 

We do not need two or more optimal basic feasible solutions to a linear 
programming problem in order to have an infinite number of optimal 
solutions. The other possibility is illustrated in Chapter 1, Fig. 1-7. In 
this ease, the objective function in its optimal position lies on a side of the 
convex set which extends to infinity. The optimal value of z is finite, but 
there are optimal solutions with arbitrarily large variables, This situation 
can also be generalized. 

Assume that we have an optimal basic solution to a linear programming | 
problem and that for some vector a; not in the basis, z; — c; = 0, and all _ 
yi < 0,17 = 1,..., m. We сап then form a new feasible solution 


} (rai — 0yi)b; + бај = b, , 82 0, (3-63) 


which ean have m +- 1 nonzero variables. The value of z for this solution — 
is the same as that for the optimal basie feasible solution; hence the solu- 
tion (3-63) is optimal. However, this holds true for arbitrarily large 6. 
Moreover, at least one y;; < 0 because aj ~ 0. Consequently, there are — 
optimal solutions for which at least two of the variables can be made - 
arbitrarily large.* Thus we have shown that if there is an optimal basic — 
feasible solution to a linear programming problem and, for some aj not in the _ 
basis, 2) — су = 0, Yi; < 0 for all i, then (3-63) is also an optimal solution — 
for any б > 0. 1 
If the set of variables yielding the optimal value of the objective function 
is not unique, we say that there are alternative optima (see Chapter 1). 


3-10 Extreme points and basic feasible solutions. Now we shall 
finally return to the geometric interpretation of linear programming. 
Recall that in Chapter 2 it was proved that the set of feasible solutions 
to Ax = b is a convex set. Here we wish to make the connection between 
basic feasible solutions and the extreme points of the convex set of feasible 
solutions to Ax = b. It will be shown that every basic feasible solution 
is an extreme point of the convex set of feasible solutions, and that every 
extreme point is a basic feasible solution to the set of constraints. 


* The reader may feel that this statement is contradicted by an example whose ` 
feasible solutions lie in the region shown in Fig. 1-6 and whose objective func- 
tion ів г = 10z. In the figure, only тә can be made arbitrarily large. However, 
recall that here slack and surplus variables are included in the discussion, AS 
x2 increases (Fig. 1-6), the slack variable (ог шү — za < 0 also increases. 
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Imagine that we have a basie feasible solution x to the set of constraints 
Ax — b. The vector x will be taken to be an n-component vector; hence 
it includes both zero and nonzero variables. Let us assume that the 
variables have been numbered so that x has the form 


х = [хв,0), xs = Bb. (3-04) 
To demonstrate that x is an extreme point, we must show that there do 
not exist feasible solutions x1, xa different from x such that x = Ax; + 


(1 = d)x2, 0 < ^ < I. Suppose that there existed feasible solutions x, 
and x, such that 


x= dx, + (1 — Nx, 0<۹ <1. (3-05) 
Write xı, xo as follows: 
xı = (ш, vı], X2 [uo Val, (8-66) 


where ш, из аге m-component vectors and уз, V2 аге (n — m)-component 
vectors, Recall that x is given by (3-64), and use (3-66) in (3-65). Then 
for the last n — m components of (3-65), we have 


0 = А + (1 — Nye (3-67) 
But À, (1 — А) > 0 and vi, va 2 0; (3-67) can hold only if 
yi = +з = 0. (3-08) 


Therefore we must have 
Ax, = Вш =b; Ax = Bu, = b. (3-69) 


However, the expression of the vector b in terms of the basis vectors is 
unique, and we must conclude that 


Xp= Uy =ч; XK =X = Xe (3-70) 


The above arguments show that there do not exist-feasible solutions differ- 
ent from x such that (3-65) holds. Consequently, x is an extreme point, 
that is, any basic feasible solution is an extreme point of the convex set 
of feasible solutions. 

Next let us demonstrate that any extreme point x* = [zı,..., zn] of 
the set of feasible solutions is a basic solution. To do this, we shall prove 
that the vectors associated with the positive components of x* are linearly 
independent. Assume that, k components of x* are nonzero; number the 
variables so that the first k components are nonzero. Then 


k 
dD zai = b, ж > Ops He Ayes. kh (3-71) 
fal 
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If the columns corresponding to the nonzero components of x* are linearly 
dependent, then there exist A; not all zero such that 


k 
Ула = 0. (3-72) 
i=1 

Now consider 


Tti ‚ 
= Tr 0 
7 mn IM , №7 0, i 


1 


AN h (3-73) 
Note that 7 is a positive number. If we select an e, 0 < є < 7, then 


ti tA >0 and Zi €, i-1d...,k. (3-74) 


Define an n-component column vector А # 0 which has the А; in the first 
k positions and zero for the last n — k components. Write 


х= х фел; x = х* – е. (3-75) 
Because of (3-74), x, > 0, x2 > 0. Furthermore by (3-72), 
AX = 0, (3-76) 
so that 
Ах, = b; Ax» = b. (3-77) 


If follows that x), xo are feasible solutions different from x*, and 


х* = $x + dre. (3-78) 


This contradicts the fact that x* is an extreme point. Hence the columns 
of A associated with the nonzero components of any extreme point of the 
convex set of feasible solutions must be linearly independent. There 
cannot be more than m linearly independent columns of A, and therefore 
an extreme point cannot have more than m positive components. If an 
extreme point has less than m positive components, it can be interpreted 
аз a degenerate basic solution, since we can imagine that some m — k 
columns of A are added at a zero level to yield a degenerate basic solution. 

We have shown that every basic feasible solution to Ax — b is an 
extreme point of the convex set of feasible solutions, and that every extreme 
point is а basic feasible solution. In the absence of degeneracy, it must be 
true that there is a one-to-one correspondence between the extreme points 
and the basic feasible solutions, i.e., there is only one extreme point for a 
given basie feasible solution, and only one basic feasible solution cor- 
responds to any one extreme point. It is important to note, however, 
that when degeneracy occurs, this is not the case. If less than m variables 
are positive, the vector or vectors from A which are added at a zero level 
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to yield a degenerate basic feasible solution need not be unique. For this 
reason, a number of different degenerate basic feasible solutions can 
correspond to the same extreme point. 

"This observation allows us to interpret geometrically what happens in 
the presence of degeneracy. If some vector with z; — c; < 0 is inserted 
into the basis and the 8 of (3-40) is zero so that it enters at а zero level, 
we obtain a new degenerate basic feasible solution which is the same 
extreme point. We have not moved away from the initial extreme point; 
we have simply changed the vectors appearing at a zero level in the basis. 
The geometric interpretation also indicates how cycling could develop. 
Although the basis would be changed by inserting and removing vectors 
at a zero level, we would never move away from the extreme point; finally 
we might obtain a new basis which, however, would prove to be our 
starting basis. The geometric interpretation also gives an intuitive sug- 
gestion as to why we should be able to avoid the cycling problem. If we 
are at an extreme point where degeneracy is present, and if there is another 
extreme point with an improved value of the objective function, then we 
expect that there will be an edge of the convex set leading from the given 
extreme point to an adjacent extreme point with an improved value of the 
objective function. This intuitive notion is correct and will be discussed in 
greater detail later. 

In this section, we have shown that every extreme point of the convex 
set of feasible solutions is a basic feasible solution. We have proved in 
Chapter 2 that if there is an optimal solution, then one or more of the 
extreme points are optimal. Hence if there is an optimal solution, one or 
more of the basic feasible solutions will be optimal (this result is completely 
independent of any nondegeneracy assumptions). Thus we have obtained 
the general and very important result that an optimal solution to a linear 
programming problem never needs to have more than m variables different 
from zero (m is, of course, the number of constraint equations). To emphasize 
the importance of this result, we note that to maximize the profit in the 
example of Section 1-3, it is never necessary to make more than three of 
the four products. 

The set of constraints (3-15) was obtained from the constraints (3-1) by 
addition of slack and surplus variables. It is very easy to show that there 
is a one-to-one correspondence between the extreme points of the convex 
set of feasible solutions to (8-15) and the extreme points of the convex 
set of feasible solutions to (3-1). Problem 3-18 will ask the reader to show 
how to obtain an extreme-point solution to (3-1) from any basic feasible 
solution to (3-15), and to prove that there is a one-to-one correspondence 
between the extreme points. 

The results of this section also prove that the convex set of feasible 
solutions has a finite number of extreme points. The number of extreme 
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points is equal to the number of different basic feasible solutions (degen- 
erate basic feasible solutions with the same values of positive variables | 
are not considered to be different). This number cannot be greater than 
т\т = m)! 1 


REFERENCES 


1. R. G. D. Arten, Mathematical Economics. London: Macmillan, 1956. 

2. A. Cuarnes, W. W. Cooper, and A. HENDERSON, An Introduction 10 
Linear Programming. New York: Wiley, 1953. 

3. 8. 1. Gass, Linear Programming. New York: McGraw-Hill, 1958. 

4. B. Karun, Mathematical Methods and Theory in Games, Programming, and 
Economics. Reading, Mass.: Addison-Wesley, 1959, 

5. T. C. Koopmans, ed. Activity Analysis of Production and Allocation. 
New York: Wiley, 1951, 

6. 8. Vaspa, The Theory of Games and Linear Programming. London} 
Methuen, 1956. 


3 


E 


PROBLEMS 


3-1. We have noted in Chapter 2 that any point in a convex polyhedron can 
be written as а convex combination of its extreme points. Leta; ..., a, be the 
k extreme points of a convex polyhedron in Е”. Any point b in the convex | 
polyhedron can be written b = Xf aj, all uj > 0, Уш = 1. Show that. 
when any point b of the polyhedron is written as a convex combination of the 
extreme points, there exist шу such that no more than m + 1 of the и; need be © 
different from zero. M 

3-2. Study the problem of reducing any solution x, which may contain both — 
positive and negative variables, of Ax = b to a basic feasible solution (provided - 
there is one). What problems are involved which do not appear in the reduction _ 
of any feasible solution to a basic feasible solution? "a 

3-3. Show that, in the absence of degeneracy, every optimal basie solution 
has z; — c; > 0 for every j. 

3-4. If an optimal basic solution is nondegenerate and 2; — су > 0 for every 
vector not in the basis, show that the optimal solution is unique. |] 

3-5. Prove that if an optimal nonbasie solution to a linear programming prob- - 
lem exists, we obtain either two or more optimal basic feasible solutions or ab —. 
least one optimal basic solution which has an edge extending to infinity. 

3-6. How many edges can emanate from any given extreme point of the con- 
vex set of feasible solutions? 

3-7. Prove that the statement, “if there is a feasible solution to Ax = b, 
r(A) = m, then there is a basic feasible solution" has its geometrical equivalent 
in the statement, “if the convex set of feasible solutions is not null, then the 
convex set has at least one extreme point.” 
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3-8. Show that if the convex set of feasible solutions to Ax = b is a convex 
polyhedron, then at least one of the extreme points must be optimal, Use the 
fact that any feasible solution is a convex combination of the extreme points. 
This is one of the theorems in Charnes’ development [2]. 

3-9. Study the problem of devising a method for inserting two vectors in- 
stead of a single vector into the basis at the same time. What problems are 
involved? 

3-10. Show that, with the proper interpretation, we can write zj; — cj = 
ðz/ðz;. Indicate clearly which variables vary and which variables remain 
constant. 

3-11. Show that, with the proper interpretation, we can write Org;/Ozy = 
—yix. Indicate clearly which variables are held constant and which vary mutatis 
mutandis (as they must). 

3-12. Consider the linear programming problem Ax = b, x > 0, maxz = cx. 
If x* is an optimal basic solution to this problem, will x* be an optimal basic 
solution to a problem whose price vector is Ac, А > 0? Will x* be an optimal 
solution to a problem whose price vector is c + M, for all A # 0? Why or why 
not? Is there any special case for which this last statement will always be true? 

3-13. Is it meaningful to have a linear programming problem for which the 
requirements vector b = 0? j 

3-14. In finding the minimum in Eq: (3-73), we did not consider only № > 0. 
Why? 

3-15. Let x > 0 be any feasible solution to Ax = b. Show that if ahy vari- 
able z; > 0 is selected, then as 2; is increased, z = сх will either remain un- 
changed, inerease monotonically, or decrease monotonically, i.e., show that z 
will never go through a maximum or minimum as а function of z; (naturally, we 
suppose that when z; is changed, the other variables are changed in such a way 
that Ax = b). Assume that we wish to maximize z, and that z increases as ту is 
increased. Show that if z is not unbounded, then for z; sufficiently large, at 
least one of the other variables must be negative. The largest possible. value of 
zj which will still maintain a feasible solution is reached when any further in- 
crease in ту would cause one of the other variables to become negative, Using 
this type of analysis, prove that if a linear programming problem has an optimal 
solution, at least one basic feasible solution will be optimal. 

3-16. Let xj = B-!b be a basic feasible solution to Ax = b, and aj a vector 
from A not in b. Then if y; = Boa; show that B(x» — Oyj) + ба, = b isa 
feasible solution to Ax = b for any 0,0 < 0 < Omax, where 


бмк = min {ys > o}: 


For which values of @ is the new solution basic? Use this result to derive the 
criterion (3-40) for determining the vector b, to be removed from the basis when 
aj is inserted. 

3-17. Reduce 2a1-+ 4а2 + аз = b to а basic feasible solution if ai = [2, 1], 
аз = [1,4], аз = [2, 0]. 
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3-18. Denote by X; the convex set of feasible solutions to Eq. (3-1) and by. 
X2 the convex set of feasible solutions to Eq. (3-15). Show that there isa 
one-to-one correspondence between the extreme points of X; and X». Also show 
how to determine the extreme points of X from the extreme points of Xo, 

3-19. Suppose that the value of іп Eq. (3-40) is positive and that the vector 
to be removed is uniquely determined by Eq. (3-40), Under what conditions 
will the new basic feasible solution be degenerate? Suppose that the current 
basic feasible solution is not degenerate. Then, with the above assumptions, 
prove that the new basic solution will also be nondegenerate. 

3-20. Find an optimal solution to the following linear programming problem { 
by computing all Базїс solutions and then finding one that maximizes the ob- 
jective function. 

211 + 312 — 23+ 424 = 8, 


х1 — 222+ бхз — 724 = —3, 
all z; > 0, 
тах г = 2x1 + 312 + 418 + 724. 


3-21. For the linear programming problem solved in Problem 3-20, compute 
the yj and z; for each a; not in. ће optimal basis. Show that the optimality 
condition is satisfied for the optimal solution. 

3-22. Consider the set of constraints Ax — b [A being mX n, n > m, 
r(A) = m] for some linear programming problem. Suppose that we have a 
feasible, but not basic, solution to the constraints. Discuss in detail the numer- 
ical procedure for reducing this feasible solution to a basic feasible solution. 
Give special attention to the way in which one can obtain a relation such as 
Eq. (3-21) which expresses the linear dependence of the columns of A, iej 
discuss how numerical values for the o; may be found. Hint: Review the ma- 
terial in Chapter 2 on solving simultaneous linear equations. 

3-23. Consider the set of equations 


211 — 322 + 4x3 + 6x4 = 25, 
zı + 2x2 + 823 — 824+ 5х5 = 12. 


Note that 212,22 = 1, 23 = 3, 5 = 2, х5 = 1 is a feasible solution. 
Reduce this feasible solution to two different basic feasible solutions. 
3-24. Consider the set of equations 
бтр — 412 4- 3r3 + 24 = 3, 
271 + 22+ 523 — 324 = 0, 
т + 622 — 4x4 + 2z4 = 15, 


A feasible solution is zı = 1,29 = 2,13 = 1, та = 3. Reduce this solution to 
а basic feasible solution, 

3-25. Consider the linear Programming problem deseribed by Eq. (1-16). 
Solve the problem by obtaining all basie solutions and selecting the one that 
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maximizes the objective function. Show the relation between the basic feasible 
solutions and the extreme points of the region shown in Fig. 1-1. For the 
optimal basic feasible solution, compute the уу, 2; for the vectors not in the 
basis and demonstrate that the optimality condition is satisfied. 4 

3-26. Make use of the basic solutions obtained in Problem 3-25 to show 
that for the linear programming problem described by Eq. (1-17), there are 
precisely two basic feasible solutions which maximize the objective function. 
Show that any convex combination of these two basic feasible solutions is also 
an optimal solution. 

3-27. Consider the linear programming problem described by Eq. (1-19). 
Solve the problem by determining all basic solutions and find the one that 
maximizes the objective function. Relate the basic feasible solutions to the 
extreme points of the polyhedron shown in Fig. 1-4. Show that adjacent ex- 
treme points correspond to basic feasible solutions that differ only in one basis 
vector, i.e., in moving from one extreme point to an adjacent one just a single 
vector in the basis is changed. 

3-28. Consider the linear programming problem presented in Problem 3-20. 
Show that a solution which involves only the variables x1, x2 at a nonzero level 
is feasible and basic. For this solution, compute the уу, z; for each vector not in 
the basis. Which of the a; not in the basis can be inserted to obtain a new basic 
feasible solution by removing one of the vectors in the basis? Find all such 
feasible solutions which can be obtained by changing a single vector in the 
basis. For which of these solutions is the value of the objective function 
increased? 


CHAPTER 4 


DETAILED DEVELOPMENT AND 
COMPUTATIONAL ASPECTS OF THE SIMPLEX METHOD 
“Tf you can look into the seeds of time, 


And say which grain will grow and which will not, 
Speak then to me. . . ." 


Shakespeare— Macbeth, 
* 

4-1 The simplex method. The time has now come to put together 
what we have learned and to develop the details of the simplex method for 
solving the general linear programming problem. After the last few Sec-. 
tions of the previous chapter, it should be fairly clear to the reader how a 
linear programming problem might be solved. f- 
We wish to solve the linear programming problem 


Ax— b, x0,  maxz-— сх, (4-1) 


where А is an m X n matrix. For convenience, the discussion will be cen- : 
tered on a maximization problem. Whenever significant conclusions are 

. obtained, the corresponding adaptation to a minimization problem will. 
also be made. 

The results of the preceding chapter have shown: (1) If there is a feasible. 
solution to a linear programming problem, there is a basic feasible solution. - 
(2) If an optimal solution exists, one of the basic feasible solutions will be 
optimal. (3) If we have a basic feasible solution which is not optimal, then, _ 
provided that degeneracy never occurs, it is possible, by changing a single - 
vector in the basis at a time (i.e., by driving one variable in the basic _ 
solution to zero and allowing another one, not in the basic solution, to _ 
become positive), to reach in a finite number of steps an optimal basic 
solution, or to show that the objective function can be made arbitrarily 
large. Here we have the essence of a computational procedure for solving a 
linear programming problem. 

Given any basic feasible, but not optimal, solution, the theory developed 
indicated how to obtain a new basic solution, with the new value of the 
objective function at least as good as the old one; or it pointed out that, by — 
adding one vector to the solution, an unbounded solution involving as 
many as т + 1 positive variables existed. In addition to the basic feasible 
solution хр, we used the y; and z; — c; for the vectors not in the basis in 
order to determine the new basic feasible solution. To find the new basic 
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feasible solution, we first select the vectors for which 2; — e; < 0. If any 
one of these vectors has all yj; < 0, i = 1,..., m, there is an unbounded 
solution. Suppose that this is not the case. Then we can insert any vector 
aj with z; — су < 0 and obtain a new basic feasible solution, with the new 
value of the objective function at least as good as the original one. We 
are not free to remove an arbitrary vector from the basis. If the new basic 
solution is to be feasible, the veetor to be removed must be chosen by 
means of (3-40). When we make the assumption that degeneracy does 
not occur, the minimum in (3-40) will be unique (fer if it is not, the new 
basic solution will be degenerate). Furthermore, if degeneracy does not 
occur, the new value of the objective function will be greater than the old 
one. The new values of the basic variables are given by (3-42), and the 
new value of the objective function is given by (3-45). 

We now wish to know whether the new basic feasible solution so ob- 
tained is optimal. This requires that we find the new z; — cj, which we 
shall denote by 2; — cj. If it is not optimal, we shall repeat the process 
and compute another basic feasible solution with a further improvement in 
the value of the objective function. To do this, we need new y;'s, which we 
shall denote by $j. (Note that all new values will be distinguished from the 
old ones by a caret.) Let us then compute the 2; — су, ў; in terms of the 
2j — cj and уу. Assume that we inserted a; into the basis and removed b,. 
In terms of the original basis, we have for any aj, 


ay = Y vids. (4-2) 


i=l 


However, we replace b, by az, and hence 


b=- X DN (4-3) 


When (4-3) is substituted into (4-2), we find 


ay = yj = yo I bi + hay = = з dbo (4-4) 
at v k tod 


dr 


where b; = by, i # r; 6, = ay. Comparison shows that 


“Ga = ws — 008, ifr, (4-5a) 
ба = Hi. (4-56) 


Equations (4-5а) апа (4-5b) indicate how to compute the у from the yij. 
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These formulas are quite similar to those of (3-42) for the new values of 
the basic variables. 
To compute 2; — cj, we use the definition 


m 
£j — cj = ёвў;— су = У) êndi — Cj (4-6) 
i=l 


However, ép; = cpi, û 7£ 7; sr = cy. Using (4-5), we obtain 


m 
Yir Yri 
ў 63 = i ج‎ HS = Ck — Cj. 4-1 
п 9 = sen (Uy) Fy E s (4-7) 
i*r 


In (4-7) it is convenient to include in the summation the term 
Vrk 


CBr (us = Vi. = 0. 


Expansion of (4-7) then yields 


m NS 
4j — с = Denys — c; — 3 Yes — ce)» 
int Vrk Niza 
or | 
ره — ز2‎ = zj — زه‎ = a (zx — св). (4-8) 
ri 


Equation (4-8) shows how to compute 2; — су from zj — cj, zy — Ck, 
and ууу, Yrk- 

"The above paragraphs contain the essence of the simplex method. It 
is an iterative procedure, and the steps can be briefly described, as follows: 

Assume that we have a basic feasible solution xg to the linear pro- 
gramming problem, with z the value of the objective function, and that, 
for all aj, the y; and z; — c; corresponding to this given basie feasible 
solution are known. 


(1) Examine the z; — cj. There are three cases to be distinguished: 
(a) All z; — c; > 0. In this case, the given basic feasible solution 
is optimal. 

(b) One or more 2; — с; < 0, and for at least опе ay for which 
2k — Ce < 0, all yj; < 0. Then there is an unbounded solution. 
(c) One or more 2; — c; < 0, and each of these has Yi; > 0 for at 
least one т, Select any one of these vectors, say а, to insert into the 
basis. 

(2) When case (1c) holds, compute the vector to be removed from the 

basis matrix B, using 


z . [BiZ 
UL min | "Wk o}. (4-9) 


Column r is then removed and replaced by ag. 
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(3) Using formulas (3-42), (8-45), (4-5), and (4-8), with the subscript 
k replacing j in (3-42) and (3-45), compute $5, £, and $j, 2; — c; for all j. 
This yields the quantities of interest for the new basic feasible solution. 
Return to step 1. 

Tn the absence of degeneracy, this iterative procedure leads to an optimal 
basic feasible solution in a finite number of steps; it is the simplex method 
for solving linear programming problems. 


4-2 Selection of the vector to enter the basis. Several points remain to 
be clarified. Note first of all that step (1c) does not represent a procedure 
which uniquely determines the vector to enter the basis. In (1c) we simply 
suggested that any vector with 2; — c; < 0 would do. If we insert aj, 
the increase in the objective function is 6(c; — 2j) = (znr/yrj)(c; — 2j). 
Hence it seems logical to select the vector to enter the basis which will give 
the greatest increase in z. According to this criterion, a; is selected by means 
of 

ey (e — э) = max m (c; — 2) 25 — с < 0. (4-10) 
This is а perfectly sound approach, and without further investigation it 
would appear to be the best criterion for selecting the vector to enter the 
basis. 

In order to apply (4-10), we must compute z5,/y;; from (4-9) for each 
ajhavingz; — c; < 0. Such computations can become quite cumbersome 
if there is a large number of а; with z; — c; < 0. Let us note that the 
change in the objective function depends on zp,/y,; and z; — cj. Another 
procedure for choosing the vector to enter the basis consists of selecting 
the largest c; — 2; while neglecting the value of z5,/y,j. This is equiva- 
lent to choosing the vector by means of 


Zk — Ch = miiia (ej — cj), $; — 0; < 0. (4-11) 


We select the vector with the smallest algebraic value (largest absolute 
value) of гу — c; for those z; — c; < 0. Both (4-10) and (4-11) have 
been applied to actual problems, with the interesting result that. (4-11) 
leads to an optimal solution in about the same number of steps as (4-10). 
In other words, from the computational point of view, there seems to be 
little or no advantage in using the more complicated criterion (4-10) to 
determine the vector to enter the basis. No computation is needed to 
apply (4-11); we simply choose the smallest 2; — cj. For this reason, 
(4-11) is usually used in preference to (4-10). In particular, almost all 
computer codes for solving linear programming problems employ (4-11). 
The method (4-10) is referred to as the method of steepest ascents, 
since it gives the greatest possible increase in z at each iteration. The 
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method (4-11) is an approximation to the method of steepest ascents. It 
will not, in general, give the greatest increase in z at each step, but on the ` 
average it seems to be about as efficient as the method of steepest ascents. | 

The use of (4-11) in determining the vector to be inserted may yield 
two or more vectors with the same minimum 2; — cj. In this case, the 
vector to be inserted into the basis is still not uniquely determined. When: 
the computations are performed by hand, one can simply choose any one 
of the tied vectors or use a chance device, such as flipping a coin, to make: 
the selection. Of course, wé could compute xp;/yr; for each tied vector and 
apply (4-10), but this would not necessarily break the tie; moreover, this 
technique requires additional calculations, and hence it is not often used, 
When the computations are performed on a digital computer, a tie i 
often resolved by selecting the vector with the lowest j-index. А 

The decisions which finally determine how many iterations there will) 
be in solving a problem by the simplex method are those which, at each: 
iteration, determine the vector to enter the basis. This is, in general, the 
only degree of freedom when one starts with a given basic feasible solution, 
The vector to leave the basis cannot be chosen arbitrarily, but must be: 
determined by (4-9). In the above paragraphs, we suggested what seems _ 
to be a reasonable procedure for determining the vector to enter the basis. ! 
It is by no means true that our criterion for selecting this vector yields a 
procedure which reaches an optimal solution in the smallest number of 
steps. The ideal criterion would be one which assured us that once a vector. _ 
had been inserted into the basis, it would never have to be removed again, - 
i.e., a minimum of basis changes would lead to the optimal basis; in fact, 
no more than m iterations would ever be needed. Unfortunately, no criteria. 
have been developed which will guarantee that once a vector has been 
inserted into the basis, it will never be removed. In the simplex method 
[which uses (4-11) or (4-10) to determine which vector should enter the 
basis], a vector may enter the basis and leave again. In fact, this can 
happen several times. 

If the linear programming problem is a minimization problem, the 
selection criteria equivalent to (4-10) and (4-11) are, respectively, 


Inr . (Lae 2 
ya = 90 min "C; гй}, £j—6; >0, (4-12) 


апа 
fà — @ = 3 (2; — су), 2 — с> 0. (4-13) 


Another simplification dealing with the vector to be inserted is usually 
introduced into the simplex method. This involves unbounded solutions. 
Instead of inspecting all vectors with гу — c; < 0 to see whether any one 
has yj < 0 for all i, the standard procedure is to examine only the yi for 
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the vector a, chosen to enter the basis, The saving in time and effort is 
particularly great when digital computers are used, because it is quite 
time-consuming to examine all components of each of a large number of 
vectors. Since we examine only the vector chosen to enter the basis, it 
may turn out that if there is an unbounded solution, more iterations will 
be required before we discover this fact. Of course, if there is an unbounded 
solution, we shall ultimately find it, However, although we run the risk 
of more iterations, the simplification discussed here does save time on the 
average because unbounded solutions caused by an incorrect formulation 
of the problem occur only rarely. 


4-3 Degeneracy and breaking ties. The degeneracy problem has been 
us for some time. By now the reader may feel that it presents a 
formidable problem, even though we have indicated that there are ways to 
avoid it (these will be discussed later). Fortunately, in practice, degeneracy 
is really no problem at all, If we use (4-11) to determine the vector to be 
inserted into the basis and (4-9) to determine the vector to be removed, 
the simplex method seems to solve any real-world problem without any 
difficulty, although a large number of the basic variables vanishes. No 
practical problem has ever cycled, and therefore degeneracy has never 
made it impossible to reach an optimal solution by means of the simplex 
method in the form discussed above. 

It has been noted бапес that or iha anaes eh Mags); the vector 
to be removed is uniquely determined by (4-9). When degeneracy їн 
present, the minimum in (4-9) may not be unique. We have already seen 
that any one of the vectors corresponding to the minimum value may be 
removed, and the new basie solution will be feasible (and degenerate). 
When performing the computations by hand, we usually make an arbitrary 
choice among the tied vectors or use a chance device to determine the one 
to be removed. When the computations are performed on a digital com- 
puter, specific rules must be followed, such aa choosing the vector with the 
largest уд, or if this does not break the tie, selecting the vector with the 
smallest index, In this way, the vector to be removed is uniquely de- 
termined, All the rules suggested here have proved quite satisfactory in 
practical applications of the simplex method. We shall later develop a 
method for uniquely determining the vector to be removed when do- 
generacy is present, which will also ensure that cycling will never occur, 
However, this procedure is time-consuming, especially on а digital com- 
puter, and rarely used in practice, because the simple rules discussed above 
are satisfactory, and cycling never seems to occur. 

In the preceding paragraphs, we have described the rudiments of the 
simplex method, Its essential feature is the change of only one vector in 
the basis at a time, The reader may have wondered throughout the past 
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discussion why we have restricted our attention to changing only a single 
vector in the basis at each iteration. The answer is very simple: No way 
has been found to change more than one vector at a time that is less time 
consuming than changing only a single vector. (See Problem 3-9.) 


4-4 Further development of the transformation formulas. It may be 
recalled that in Chapter 2 we presented a technique for computing the ` 
inverse of a matrix which differs by only one column from another matrix 
whose inverse is known. In the simplex method, only a single vector in 
the basis matrix is changed at each iteration. It is instructive and inter- 
esting to look at the problem of developing the transformation formulas 
in the light of the theory discussed in Chapter 2. 

We begin with a basic feasible solution characterized by the basis matrix 
В = (bi,..., bm). Now we remove column r, that is b,, and replace it 
by column а. Let us assume that we know B7", Then for the original 
basic feasible solution x; = B^!b, y; = Ва, j = 1, ..., n. If the 
new basis matrix is B, it follows that $5 = B-'b, 9; = Bla, are the new 
values of xs, yj. Chapter 2 has shown us how to compute B^! from B7, 
We know that 

B- = ЕВ-!, (4-14) 
where E is an m X m matrix which differs from the identity matrix only 
in that the rth column of E contains the vector 

JR... lk, LE... LM. a 
ТУ E СА. a Gm 
Hence we can write 


$5, = ЕВ-'Ь = Exp; $;— EB-!a; = Еу, (4-16) 


1f (4-16) is reduced to component form, we simply obtain the transforma- 
tion formulas (3-42) and (4-5). 

It is sometimes convenient to derive another form of the transformation 
formulas for xg, y; by means of (4-16). Note that we can write 


E=1+F, (4-17) 


and F is a matrix which differs only in its rth column from a null matrix. 
The rth column of F is the vector ¢ = у — e,. Thus, 


F = (0,...,0,¢,0,...,0), фіп column r, (4-18) 
and 
wa) D an Yık Yak d Yr+ik Ymk 
-29—e-|—P/Ux,...,— V1, Dy прев ,_ Vue]. 
7 ^ [ yn van ара FA 


(4-19) 
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With this notation, (4-16) becomes 


£5 = (I + F)xs = хр + Fxg. (4-20) 
But F has zeros everywhere except in the rth column; thus 
Fx = 25,9. (4-21) 
Substitution of (4-21) into (4-20) gives 
$5 = Xp + ray. (4-22) 
In the same way, we find 
3; = yi + rsd. (4-23) ° 


Equations (4-22) and (4-23) are particularly simple to use. Note that 
the transformation of xg and of all the y; depends only on their original 
values and on the vector $ which in turn depends only on yy. Once ф is 
computed, the $5, ў follow immediately from (4-22) and (4-23). 

Another very interesting observation can now be made. Suppose that 
we write 


tai = уо (i= 1,...,m); B= edi) o 2j — Cj = Vm 
(4-24) 
* 
The transformation equation for z, that is (3-45), can then be written 
feit = Veto Yro [> "nt (4-25) 
and the transformation equation (4-8) for the z; — c; becomes 
Invi, = Vni + Wi [= ты, (4-26) 


Now if we define Y; = [yij,-++,¥m4is),J = 0} ... , n, the transforma- 
tion of every quantity of interest can be expressed in terms of the single 
transformation formula: 


$;2 Y; ue, ја 0,1,...,, (4-27) 
where 


Ү; = [yj ут+1,3), j = 0,...,п; e -[s- хаи): 
The Ү;, Ф are (m + 1)-component vectors, The single vector equation 


(4-27) yields the transformation relation for all quantities of interest. For 
clarity, let us restate the meaning of the Y; and Ф in terms of the more 
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familiar variables. We have 


Yo = [va1,---, Bm, 2] = Brio -- +) Ym+i.0), (4-28) 
Y; = l5... inp 8j — 6] — lys O Ymi аз P= 1 m) 
(4-29) - 
o [ йк |, Yok, 1 cq вен Unei, r а]. 
Yrk Yrk Ук Yrk Yrk Yrk р 
(4-30) 
The transformation formula (4-27) is useful for both manual and machine 
computations. 


The above definitions can also be used to simplify the transformation. \ 
formulas (3—42), (3-45), (4-5), (4-8), so that all these results can be 
written in the compact form 


$e = yug — уа 7, 3209,...,^h, 4-—1...,m--l1 {эё т, 
Yrk 

(4-318) — 

Oi = 005 m 0n (4-11) _ 


Equations (4-31a) and (4-31b) are merely a rearrangement of the com- _ 
ponent form of (4-27). They can also be obtained directly by substituting 
the new definitions into the defining equations. Incidentally, it might be — 
noted that we have encountered transformation formulas of this type _ 
in Chapter 2 in the discussion of the gaussian reduction technique for 
solving simultaneous linear equations. 1 
In this section, we have shown that all quantities of interest in the 
simplex method transform in the same way in moving from one basic _ 
solution to the next. The general transformation equation can be written. | 
in vector form as (4-27) and in component form as (4-31). 


4-5 The initial basic feasible solution—artificial variables. Up to the 
present time, all of our discussions have been based on the assumption that. ' 
we had an initial basic feasible solution to the linear programming problem. 
We have yet to treat the problem of finding an initial basic feasible solu- 
tion. Once this has been done, we shall have a computational procedure 
which will in theory solve any linear programming problem. Now it may 
turn out that, because of improper formulation, our linear programming 
problem does not have any solution, In this section, we wish to develop & 
general procedure which either leads to a basic feasible solution or shows 
that the problem has no feasible solution. 
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We shall begin by considering a case where a basic feasible solution can 
be immediately determined. Suppose that in the original formulation of 
the problem each of the inequalities was of the form <, so that a slack 
variable was added to each constraint to convert it to an equality. 'The 
matrix A for the set of constraints Ax = b then has the form 


А = (Е,1), (4-32) 


where I is an mth-order identity matrix. This follows because the column 
corresponding to the slack variable z,4,; is e; If we write х = [х,, Xs], 
where x, contains the m slack variables and x, contains the original r 
variables, then, setting x, — 0, we have 


Ix, = b. (4-33) 


Here we have a basic solution involving only the slack variables. Further- 
more, it is feasible because x, = b, and b > 0 by virtue of the manner 
originally chosen to set up the problem, This basic solution is especially 


easy to work with since the basis matrix B = I, and B~' = I. Therefore, 
yS Bm aj Sap айал! gas TP fni (4-34) 

and 
cn — 0 (4-35) 


because the prices associated with slack variables are zero. Consequently, 
£j/— Cj; €BYj — CHE Th (4-36) 
2 = свхв = 0). (4-37) 


For this basic feasible solution, no computations are needed to obtain the 
quantities of interest, namely, xg, 2, yj, 2) — су. The iterative procedure 
of the simplex method can now be initiated without any difficulty. 

In the above paragraphs we have shown that when a slack variable 
appears in every constraint, it) is m trivial task to find an initial basic 
feasible solution together with the other quantities of interest, 2, уу, 2; — cj. 
However, this really did not depend on the addition of a slack variable to 
every constraint, In fact, the above procedure ean be used whenever an 
m X т identity matrix appears in A. Of course, if the columns of the 
identity matrix do not correspond to slack variables, then (4-35) may not 
hold, However, the z; — c; can still be easily computed from 


сву; — Cj = Cpaj — Cj. 


In many cases no identity matrix will appear in A. This will almost 
always be the case when some constraints do not require the addition of a 
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slack or surplus variable, Under such conditions, there is usually no ea 


efforts on attempting to construct such a solution if one exists. We have 
noted that it is desirable to have an identity matrix as the initial basi 

matrix. Suppose that we begin always with an identity matrix for the 
initial basis matrix. Instead of the original set of constraint equations 
Ax — b, let us consider a new set of constraint equations 


Ax + Ix, = (a, D[¥] =». 


We have augmented the given constraint equations by m addition 
variables Taj, with the columns corresponding to these variables being e 
"These new variables are called artificial variables (artificial because they 
have no meaning for the original set of constraints). The column ej 
which we shall sometimes designate by q;, corresponding to the artificial 
variable za; is called an artificial vector. ) 

What have we gained by pulling this trick? We have gained what we 
were looking for, since an identity matrix appears in (4-38). We then 
immediately have a basic feasible solution to the set of constraints (4-38), - 
namely x, = b,x = 0. Note that this is not, however, a feasible solution 
to the original set of constraints. Any solution to (4-38) which is also & 
solution to the original set of constraints must have x, = 0, that is, all 
artificial variables must vanish. Then (4-38) reduces to the original set of 
constraints Ax = b. The basic feasible solution x, = b to (4-38) will be 
of little use unless we can find a way to move from it to a basic feasible 
solution to the original problem. 

An interesting idea now suggests itself: Why not use the simplex method 
itself to insert legitimate columns a; from A (we use the term “legitimate” 
to distinguish the columns of A from the artificial vectors q;) into the iden- 
tity matrix in the usual step-by-step fashion and thus, by removing the 
artificial veetors from the basis, drive all the artificial variables to zero? 
In this way, we should end up with a basis containing only vectors from A, _ 
that is, with a basic feasible solution to the original set of constraints. The 
simplex method can then be continued to yield an optimal basic feasible 
solution to the original set of constraints. This is precisely what we shall do, 

In the simplex method, the z; — c; determine which vector is to be 
inserted into the basis. Thus far we have said nothing about the prices to 
be associated with the artificial variables. What we must do is assign 
prices to the artificial variables which are so unfavorable that the objec- 
tive function can be improved so long as any artificial variable remains in 
the basic feasible solution at a positive level. If z is to be maximized and 
we assign an extremely large negative price to each artificial variable, then 
we would expect that z could be improved so long as any artificial vector 
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remains in the basis at a positive level. Similarly, if z is to be minimized, 
a very large positive price should be assigned to each artificial variable, 
Let ca; be the price corresponding to the artificial variable zaj. Then we 
write 
Cai = —M, M >0, if zis to be maximized, (4-39a) 


Cai M, М> 0, if z is to be minimized. — (4-39b) 


For hand computations, M is usually not specified as a particular number. 
It is merely entered into the computations as M, and it is assumed to be 
large enough so’ that any price corresponding to a legitimate variable is 
completely negligible with respect to M. For machine computations, the 
value of M is often taken to be about 1000 times the largest price corre- 
sponding to any legitimate variable. 

If for some reason (such as the addition of slack variables to certain con- 
straints) part of an identity matrix is present in A, then it is only neces- 
sary to add the artificial vectors e; which are needed to give a complete 
identity matrix. For example, if e, appeared in A, then an artificial vector 
e, would not be added to the system. 

The introduction of artificial variables provides an easy starting point. 
for solving any linear programming problem. By applying the simplex 
method to the augmented problem, we ultimately obtain a basic feasible 
solution to the actual problem if there is one, and then, by continued appli- 
cation of the simplex method, an optimal basic feasible solution to the 
original problem is found. Initially, the artificial variables are non-nega- 
tive; because of the way the simplex method works, no artificial variable 
will ever become negative. Thus z,, > 0 always. Finally, we must have 
toi = 0 for all í if we are to obtain a feasible solution to the actual set of 
constraints, 

Given any linear programming problem 


Ax = b, x20, тах 2 = cx, (4-40) 


we can always start our computation with the augmented problem (recall 
that 1 is the sum vector), 


Ax-FIx,- b x20, x20, maxzecx— Mix, (4-41) 


and arrive immediately at the basic feasible solution x, b to (4-41). 
Application of the simplex method to (4-41) ultimately leads to an optimal 
basic feasible solution to (4-40) if such a solution exists. 

Once an artificial vector has left the basis, it has served its purpose, and 
we can forget about it. An artificial vector is never considered for re-entry 
into the basis. The reader is asked to prove in Problem 4-6 that this pro- 
cedure is legitimate. 
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The method of assigning a large negative price to an artificial vector in 
order to drive it out of the basis was first suggested by Charnes, Later we 
shall discuss another procedure for handling artificial variables known as 
the “two-phase” method. 


ExaMPLE: We shall illustrate the introduction of artificial variables in 
the following linear programming problem: 


32, + 422 + 523 + бх; 5, 
2х1 + 622 + 235 + 5x4 > 6, 
&1 ta töz t 24 = 7, 


Ti) To, 29, T4 > 0, 


< 
e 


maxz = 821 + 2х2 +23 + 5z4. 


After adding slack and surplus variables, we obtain the constraint equations 


3 450641 0 5 
2 6 1 5 0 =H [ti zo, zo, za, 25, zo] = |6|- 
I'1' 8$ FO 0 7 


There is present in A the vector e;, i.e., the first column of an identity 

matrix. We must then add two artificial vectors qı = €z, q2 = ез whose 

corresponding variables are x41, аз. The price associated with each arti- 

чн variable is —M. "Thus the augmented linear programming problem 
'omes 


$456 0100 


2 6 1 5 —1 0 1 Oi, zo, x3, 74, 26) Z5, al, Taa] 
b 1351 0001 


1 
ч © с 


x20, x20, 
тах 2 = 3r, + 2r; + та + 5r, — Mza — Mzi. 
An immediate basie feasible solution to thé augmented system is 
xa = [zs Zan teal =, Th 1 
with ч 
В = В-! =1,. B= (850,9). 


Then for all j, y; = aj Two artificial vectors with prices —M are ~ 
in columns 2, 3 of the basis. Therefore cg = (0, —M, —M). Since 
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2j — cj = сву; — су, we find 

2 —¢; = —3M —3; 1) — сә = —7M —2; 

23 — cg = —6M = 1; £4 = = —6М — 5; 

26 — ĉo = M. 


We assume that M is much larger than any other price. Consequently, 
only terms containing M count. The vector to be inserted into the basis is 
the one with the smallest 2; — cj; < 0. This is ay. The vector to be 
removed is determined from 


min [^t yi > of = nin (5,0,7 й 
The minimum is taken on for i = 2; this implies that column 2 of B or qı 
is removed and replaced by аз. Hence we see that, in the first iteration, 
one of the artificial vectors is removed from the basis and is replaced by a 
legitimate vector. 

It is by no means true that the artificial vectors will always be removed 
from the basis before a legitimate vector is removed. If, in the first con- 
straint, bı had been 2 instead of 5, the vector to be removed would have 
been found from min (2/4, 6/6, 7/1) = 1/2. The minimum is taken on 
fori = 1, and it follows that as would have been removed and replaced by 
аз. Both artificial vectors would have remained in the basis. 


4-6 Inconsistency and redundancy. The addition of artificial variables 
allows us to eliminate another problem which concerned us at the outset, 
It will be remembered that we assumed r(A) = r(A») = m, so that we 
knew there was a baste (although not necessarily feasible) solution to the 
set of constraints, Whenever we begin with an identity matrix in the set 
of constraints, it is immediately clear that а basic feasible solution exists, 
When a slack variable is added to every constraint equation of the original 
problem, then it is at onee obvious that r(A) = r(Ai) = m and, in addi- 
tion, it is clear that ther’ is a feasible solution, If artificial variables are 
added to the problem, it is also true that there is а basic feasible solution 
to the augmented system. Hence the assumption made in the initial 
discussion of the theory of the simplex method will always hold when we 
begin to solve a linear programming problem, We wish to show that if we 
start with the augmented system whose initial basic feasible solution con- 
sists entirely or in part of artificial variables, we shall ultimately determine 
by means of the simplex method whether the original constraint equations 
are consistent, and whether any one of the original constraint equations is 
redundant, 
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It suffices to consider the possibilities when the optimality criterion is 
satisfied, i.e., when all z; — с; > 0, since, if no basis is ever repeated, we 
shall ultimately reach a state in which the optimality criterion is satisfied, 
provided there is no unbounded solution. We make this assumption here, 
There are then three cases to be distinguished: 

(1) There are no artificial vectors in the basis. 

(2) One or more artificial vectors are in the basis at a zero level. 

(3) One or more artificial vectors are in the basis at a positive level. 
Each of these cases will be considered in turn. 

If there are no artificial vectors in the basis and the optimality criterion 
is satisfied, it is clear that an optimal basic feasible solution to the actual 
linear programming problem has been found. Hence we know that the 
original constraint equations are consistent and none of the equations is 
redundant. 

Let us next study what happens when one or more artificial vectors 
appear in the basis at a zero level and the optimality criterion is satisfied. 
Since all artificial variables are zero, we have a feasible solution to the 
original set of constraints, and it follows that the original constraint equa- 
tions are consistent. There still remains the question of redundancy in the 
original constraints. Two alternatives may arise: Either Vi; = 0 for all a; 
and for those т corresponding to the columns of the basis containing 
artificial vectors at a zero level, or y;; * 0 for one or more a; and for 
one or more i corresponding to the columns of B containing artificial 
veetors. 

Assume that for some j, y;; = 0 (i corresponds to a column of B con- 
taining an artificial vector). We allow yij to be either positive or negative. 
Then we know that the artificial vector can be removed from the basis and 
replaced by a;, and a basis will still be maintained. Furthermore, since the 
artificial variable was at a zero level, a; will enter the basis at a zero level, 
and the new basic solution will be feasible. The value of the objective 
function will be unchanged. If this process can be continued until all 
artificial vectors are removed, we have obtained a degenerate basic feasible 
solution (which is also optimal) involving only the columns of A. Con- 
sequently, none of the original constraint equations is redundant. 

1f the above procedure does not remove all artificial vectors, we must 
ultimately reach a state where y;; = 0 for all a; and all 7 corresponding to 
the columns of B containing artificial vectors at a zero level. Then we are 
sure that we cannot remove any artificial vector and replace it by some 8; 
while still maintaining a basis. Tt is also evident that under such circum- 
stances every column in A can be written as a linear combination of the 
columns of A in the basis. The artificial vectors are not needed to express 
any column of A in terms of the basis vectors. Suppose that there are k 
artificial vectors in the basis at a zero level. It follows that every column 
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of A can be written as a linear combination of the m — k linearly inde- 
pendent columns of A in the basis. Hence r(A) = m — k, and k of the 
original constraints were redundant. 

We can also determine which of the constraints are redundant. Suppose 
that a single artificial vector e, appears in column s of the basis at a zero 
level, and for all legitimate vectors, y,; = 0. We know that r(A, ej) = m 
and r(A) = m — 1. Denote the rows of A by a’ and the rows of (А, ej) 
by (a, 0), i = h, and (a^, 1) for the Ath row. Since r(A) = m — 1, there 
exist a; not all zero such that 


We now claim that a, = 0, and therefore а^ can be written as a linear 
combination of the other rows of A. Assume on the contrary that a, = 0. 
Because r(A, ej) = m, the only ^; satisfying 


m 
D (5,0) + (2*1) = 0 
Ez 
are à; = 0 (i = 1,...,m). However, if we set А; = ar, а contradiction is 
obtained. Applying this reasoning step by step, we see that if a number of 
artificial vectors e; appear in the basis at a zero level and the corresponding 
ys; = 0 for all j, then for each 7 the ith constraint іп the original system of 
equations is redundant and can be dropped. The solution in case (2) is of 
course an optimal solution regardless of whether the artificial vectors can 
be removed from the basis. 

We still have to discuss the case where one or more artificial vectors 
are in the basis at a positive level and the optimality condition is satis- 
fied. It is immediately evident that, in this instance, there is no feasible 
solution to the original problem; for if a feasible solution existed the arti- 
ficial variables could be driven to zero, thus yielding an improved value of z. 
This would contradict the fact that the optimality criterion is satisfied. The 
absence of a feasible solution may arise because the constraint equations 
are inconsistent or because there are solutions, but no feasible solutions. 
It is interesting to note that the simplex procedure allows us to differen- 
tiate between these two possibilities. 

If for some j there is a уу > 0 for 7 corresponding to a column of B 
which contains an artificial vector, let us insert а; into the basis and 
remove the artificial vector. The new solution may or may not be feasible. 
Such a situation could not occur if there were only a single artificial vector 
in the basis at a positive level, since then 2; — c; = —My,; + small 
terms < 0, and-the optimality criterion would not be satisfied. After this 
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insertion is made, the 2; — с; for the new basic solution can have any sign. 
We proceed to insert vectors a; and remove artifieial vectors so long as 
there is a y;; > О in a position ї corresponding to an artificial vector in the 
basis. 

The above procedure can end in only one of two ways. Either we remove 
all the artificial vectors and therefore obtain a basic but not feasible solution 
to the original set of constraints, or we reach a point where y;; < 0 for 
all j and for 7 corresponding to the columns of B which contain artificial 
vectors. Suppose that for some j, y;j < 0, and i corresponds to a column 
of B containing an artificial vector. We can insert а, and remove the 
artificial vector while maintaining a basis. The new basic solution will 
not be feasible, because a; enters at a negative level. If this process can be 
continued until all artificial vectors have been removed, we have obtained 
a basic, but not feasible, solution to the set of constraints. 

In the last case to be studied ууу = 0 for all j and for i corresponding to 
the columns of B which contain artificial yectors. Assume that there are k 
legitimate vectors from A in the basis. Then every column of A can be 
written as a linear combination of the k columns of A in the basis, and 
r(A) = k. On the other hand, b cannot be written as a linear combination 
of the legitimate vectors from A in the basis. One or more of the artificial 
vectors are also needed, This implies r(A) = k + 1, and the original 
constraint equations are inconsistent. 

Let us summarize the results of the preceding paragraphs: 


(1) If no artificial vectors appear in the basis and the optimality cri- 
terion is satisfied, then the solution is an optimal basic feasible solution to 
the given problem. The constraint equations are consistent, and there is no 
redundancy in the constraint equations. 

(2) If one or more artificial vectors appear in the basis at a zero level 
and the optimality criterion is satisfied, then the solution is an optimal 
solution to the given problem. The constraint equations are consistent, 
but in this case redundancy may exist in the constraints. 

(3) If one or more artificial vectors appear in the basis at a positive 
level and the optimality criterion is satisfied, the original problem has no 
feasible solution. There may be no feasible solution either because the 
constraint equations are inconsistent, or because there are solutions, but 
none 18 feasible. 

We have not discussed the meaning of having artificial vectors in the 
basis and an unbounded solution. This case is not of great interest, and 
the details of the analysis are left for Problem 4-23. 


4-7 Tableau format for simplex computations. When simplex calcula- 
tions are to be done by hand, it is very desirable to follow a well-organized 
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procedure, A very useful tabular form displaying all the quantities of 
interest has been developed by Orden, Dantzig, and Hoffman. Its sche- 
matic arrangement is shown in Table 4-1. Such a format is called a tableau, 
and a new tableau is constructed at each iteration, i.e., each time a new 
vector is introduced into the basis. 

The first column of the tableau gives сл, the prices corresponding to the 
vectors in the basis. ‘The second column tells which vectors are in the 
basis. Thus b, is the vector from A (or the artificial vector) which is in 
the first column of B, ete. If ayo is in the second column of B, then we 
would write ау; where by appears in Table 4-1. The third column of the 
tableau, under the heading "b," gives the current value of xp, together with 


the value of the objective function for the basic feasible solution described , 


by the given tableau. The remaining columns list the values of y; for all 
vectors in A and any artificial vectors which may have been added. We 
write B^'q; = улі The last entry in each of these columns gives 
z; — cj. The first row of the tableau gives the prices associated with 
the vectors. 

This tableau form allows us to keep track of all necessary data, It is 
also very useful in the actual performance of computations by means of 
the simplex method. Recall that we must know which column of A is in 
column î of the basis in order to determine the variable ту corresponding to 
т. Column 2 allows us to do this, To find the variable z; correspond- 
ing to tai, we look at the row in column 2 where хр; appears, and вее 
which vector this variable corresponds to. In the same way, the c; cor- 
responding to ên; is found. For example, if some particular problem has а 
three-dimensional basis, and if at some iteration ag is in column 1 of B, 
az in column 2, and ag in column 3, then, in the column “vectors in basis,” 
we write ag in the first row, az in the second row, and ag in the third row. 
The number appearing in row 1 under the column headed “b” will be 
Xe = 231, in row 2, ха = 282, and in row 3, 28 = zgs. Now imagine 
that in the next iteration аз is removed and a; inserted into column 2 
of the basis. In the new tableau, under “vectors in basis,” we shall then 
have ав, ау, ag, in that order, and the numbers in the first three rows 
under column "b" will be the values of zs, 27, rs. 

Since artificial vectors are never considered for re-entry into the basis 
once they have been removed, columns fon these vectors need not be 
included in the tableaux. "Their inclusion would only lead to unnecessary 
computational efforts in the process of transforming the vectors. However, 
if it is desired at each stage to have available B7’ for the current basis B 
(see Problem 4-11 and Chapter 11), or if the techniques described in 
Chapter 6 are being used to resolve the degeneracy problem, then it is 
necessary to include columns for the artificial vectors and to transform 
them at cach iteration. Columns for these vectors are shown in Table 4-1. 
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4-8 Use of the tableau format. Let us next see how the tableau is used 
in solving a linear programming problem. Assume that we are given a 
basie feasible solution for the problem and the tableau in the form of 
Table 4-1 for that solution. First of all, we examine the last row. If all 
2; — cj > 0, the basic feasible solution is optimal. When this is not the 
case, we select the column having the smallest z; — c; (smallest algebraic 
value—largest absolute value). Call this column k. In the event of a tie, 
use any one of the methods discussed previously to resolve the tie. If 
иф < О for all ї, there is an unbounded solution. Assuming that at least 
one у > 0, we shall insert ар into the basis. The vector to be removed is 
computed from 


TBr in (TBE 
= yi о 4-42 
Yrk Weg А ae } ( ) 


Column r of the basis is then removed. This means that the vector in the 
rth row of the column “vector in basis" is removed and replaced by ay. 
It is interesting to note that the values used in the computations of (4-42) 
are easily found. All у are found in the column headed “аһ.” The xp; 
associated with y; is found in the same row as yix in the column headed "b." 
If there is a tie for the vector to be removed, it can be broken in one of the 
ways discussed earlier. We have now found the vector to enter and leave 
the basis. 

Next it is necessary to compute a new tableau. Practically every quan- 
tity in the tableau will change in moving from one iteration to the next. 
It is the caleulation of the new tableau that requires the major amount of 
computational work. ‘For a large tableau, it can be an extremely laborious 
job. In Section 4-4 we presented two different, forms of the same trans- 
formation formulas which can be used to carry out the computation. 
First let us discuss (4-31), which is the usual form of the transformation 


formulas. Repeating (4-31), we have 


ij = уу — vine 

j = 0f. i=l. mtl, ту (443a) 
FANE LAP (4-43b) 
Ors Yrk 


Recall that yio = Bi, Ym+1,0 = % Um414 = 23 — Су. Every number in 
the tableau can be transformed by means of (4-43). Also, all the informa- 
tion needed to obtain the new tableau is included in the current tableau. 

The subscript ¢ in (4-43) refers to а row in the tableau. Row i is the ith 
row below the headings. Every row except row r transforms according to 
(4-43a). Row r is a special case and transforms according to (4-43b) ; 
for this reason, it is a good idea to begin computing the new tableau by 
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first computing row r. Note that to compute the new row r we need only 
the elements of the old row r. Each element in the new row r is found by 
dividing the corresponding element in the old row r by уук, the element in 
row r of the vector to enter the basis. The reader may find it helpful to - 
encircle in the current tableau the column k of the vector to enter the basis 
and the row r of the vector to leave the basis. We have now filled in the 
rth row of the new tableau, using only elements in the encircled row r of | 
the current tableau. 

Every other row in the new tableau is found by (4-43a). At first glance, — 
this looks like a complicated equation because so many subscripts appear. | 
In actuality, it is easy to use and to remember if one keeps in mind the 
graphic illustration of Fig. 4-1. To compute fij, we start out in the current 
tableau at column j and row i. We write down ууу; then we move along 
row û to column k (the encircled column of the vector to enter the basis) and 
write yi; — ya. Next move along column k to the encircled row r (row of 
vector to be removed) and write уу — Yik/Yrs Finally, in the encircled 
row r move back to column j and write jij = yij — (Yin/Yre)Yrj- This 
represents an easy: method of writing down the numbers which are used in 
computing j; Because of the circular path transversed in obtaining these 
numbers, we shall refer to the use of the transformation formulas (4-43) 
as the “ring around the rosy” method. 

The above procedure for filling in rows i # r can be simplified further 
by filling in the entire row i before moving on to the next row. Note 
that yik/Yre = a is constant for a given row т. Thus to find j;;, we first 
subtract ayr; from yij, and yr; is in the encircled row r and column j. Once 
« is determined for row i, we need deal only with two elements in column j 
to find ij. 

An alternative technique of transferring the tableau is the use of the vec- 
tor transformation formula (4-27). Repeating it along with the appro- 
priate definitions, we have 


25 = Yi fy,  2—0...,n (4-44) 
Үз = (yu. а Ym at aly 


PS [- PI RANDE و‎ E اا‎ iia]. 
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To use this form of the transformation formula, we first determine Ф from 
column k. Then any new column is found by adding to the corresponding 
current column y,/®, and yr; is the rth element of the old jth column (the 
element in the encircled row). Whether one prefers the "ring ground the 
rosy" method or the vector transformation formula is largely a matter of 
taste. With the simplified method of filling in all of row ї, the "ring around 
the rosy" method and the vector transformation method require precisely 
the same number of mathematical operations. 

When computing a new tableau, we shall find it helpful to recall that if 
a; is in column i of the new basis, then j = e; and 4; — ej = 0. For this 
reason we can, without any computations at all, fill in m of the columns in 
the new tableau. 

The final step in the computation of the new tableau consists in changing 
the “cp” and “vectors in basis” columns. Only the rth entry in each of 
these columns is changed. ‘The number cpr is replaced by ск, and the vector 
in the rth position under “vectors in basis” is replaced by ax. Instead of 
doing this last, the reader may prefer to perform these two changes first. 

Once the new tableau is obtained, we check to see whether the new solu- 
tion is optimal. If it is not, we repeat the whole procedure all over again. 
When an optimal solution is obtained, it will be observed that the final 
tableau provides us with all the necessary information about this optimal 
solution. Under “vectors in basis,” we see which vectors and variables are 
in the optimal basic feasible solution. Under the column headed “b,” we 
find the values of the optimal basic variables in the first,m positions. In 
the (m -+ 1)-position, we find the optimal value of z, the objective function. 

Since we almost always start out with an identity matrix, it is very easy 
to obtain the initial tableau. For this tableau xs = b, yj = aj. Usually, 
only slack or artificial variables will be in the basis, so that the price egi 
will be either 0 or +M. If there are only slack variables in the basis, 
2=0, 2 — ej 6. When some artificial vectors appear in the 
basis, z, z; — c; must be computed from z = CaXg = cab; zj — ej = 
cay} — cj = свај — ĉj. This can be done by multiplying the elements 
in the same row and columns св, b, or aj. 

When simplex computations are presented in tableau form, zeros are 
often not written in explicitly. The matrix A will quite frequently contain 
a large proportion of zeros, and their omission saves a considerable amount 
of writing. For the tableaux presented in this book, all the zeros are written 
in explicitly in order to avoid confusion. 

After having presented the tableau form convenient for manual computa- 
tions, the reader may have several questions in mind about the rationale 
of our method. As a trivial point, he may wonder why z is computed for 
each tableau, although we are interested only in the optimal value of 2; 
could we not compute z from z = ‘CBXB after obtaining the final tableau? 
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It is quite true that we do not have to compute z for each tableau. How- | 
ever, sometimes it is interesting to see how much better the optimal z is 
than the z found in some of the other tableaux. For this reason, if for no _ 
other, it is convenient to determine z at each iteration. р 

A much more profound question concerns the rationale of transferring — 
all the y; at each iteration. If one thinks about it, only хв, all the z; — cj, — 
and y, are needed at each iteration. Note that only ур is required to com- _ 
pute the vector to leave the basis. The reader may feel that the new _ 
2j — cj cannot be computed without a knowledge of all the y;. However, _ 
this is not true since 2; — c; = egy; — c; = свВ aj — с. Conse- 
quently, if the new inverse matrix B7 is known, one can find all the 
2j — cj without knowing the уу. Hence it would appear that instead of 
transforming all the y;, we could transform the basis matrix only and 
compute eyerything else from it. This is indeed precisely what the revised _ 
simplex method does. We shall later discuss this alternative procedure and _ 
compare it with the version of the simplex method developed here. His- 
torically, the simplex method, as presented here, evolved first. It also 
seems true that for hand computations most people still prefer to use this 
form of the simplex method rather than the revised. simplex method which 
we shall present later. 


4-9 Conversion of a minimization problem to a maximization problem, 
Through all the discussion of the simplex method, we have treated maxi- 
mization and minimization problems as if they were completely distinct, 
"This was done only for the purpose of showing all the pertinent relations 
for both types of problems. In actuality, any minimization problem can be 
converted by a trivial transformation into a maximization problem, and 
vice versa. 

Suppose that we have any function of n variables, f(x1,..., a»), and let 
f* be the minimum value of this function for points in some closed region 
of E". Let x* be the point (or one of the points) where f takes on its mini- 
mum value. By definition of an absolute minimum, we have for every point 
x in the region 


Ime TS O (4-45) 
Multiply (4-45) by (—1). This gives 
(-/5 — (=f) > 0 (4-46) 


for every x in the set. Therefore by definition of an absolute maximum, 
(—/*) = max (—f] (4-47) 


for points in the region. Then —f takes on its maximum value at xf. 
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Consequently, we conclude that, 
min f = f* = —(—f*) = —max (—f). (4-48) 


The minimum of a function f for some set of points is the negative of the 
maximum of the function —f. The minimum of f and the maximum of —f 
are taken on at the same points. 

Asan example suppose that f can take on the integral values (5, 4, 3, 2, 1). 
The minimum of f is 1. But —f can take on the values 


(—5, —4, —3, —2, —1), 


and the maximum of —f is —! or —max (—f) = 1 = min f. 

We now see how to convert a linear programming problem in which z is 
to be minimized into а maximization problem. It is only necessary to 
change the sign of every price. Thus 


minz = —max(—cx) = —max (—c)x. (4-49) 
The function to be maximized isz* = (—c)x. Finally, min z = —max z*. 
Suppose that we wanted to minimize 2 = 82, + 222 + zs. The corre- 
sponding maximization problem would maximizez* = —8x, — 202 — 13. 


Let us show that the simple change in sign of the prices does reverse the 
criteria for selecting the vector to enter the basis and the optimality 
criteria. It will be recalled that in the simplex method a minimum has been 
reached if 

2j — ej = cy; 4 SO, all. 


The conversion to a maximization problem replaces cy by —cj, во the 
above optimality condition becomes on multiplication by —1 ! 
(—egy; — (=e) > 0, айу; 


this is precisely the optimality criterion for a maximization problem. 
Similarly, in a minimization problem, the vector chosen to enter the basis 
4 7 
is 

th — e = max (z; — C) 2; — с> 0. 


When с; is replaced by —c;, we have 
а — (e) = min[-z— 5 (6) «0, 
J 
which is the criterion for choosing the vector to enter the basis in a maxi- 


mization problem. This shows us that the same sequence of vectors will 
enter the basis whether the problem is treated as a maximization problem 


or a minimization problem. 
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Since a minimization problem can be converted to a maximization prob- 
lem by merely changing the sign of the prices, one can, without loss of 
generality, study only maximization problems. It сап be assumed that 
any minimization problem has been converted to a maximization problem. 


4-10 Review of the simplex method. We have presented a rather long 
discussion of the simplex method for solving the general linear program- 
ming problem. It may now be worth while to give a brief review of the 
method and outline a step-by-step procedure for finding a solution. 

(1) We are interested in maximizing the linear objective function 
z = Y cjr; subject to m linear inequalities (or equalities) of the form 


r 
Das = 2), i-l..,m, (4-50) 
j=l 

and the non-negativity restrictions z; > 0 (j = 1,...,7). 


First we make sure that all b; are non-negative. If necessary, we multiply 
an inequality by —1 to obtain b; > 0. Next, slack or surplus variables are 
added to convert each inequality to an equality. Then 


2 
33 ogni sh (4-51) 
j=l 

becomes 

r 
Уа totai 0s | بب‎ 0, (4-52) 
j=1 

and 

r 
59 air; > b; (4-53) 
j=l 

becomes 

r 
DE аж; — т, = Dy + = 0, (4—54) 
jel 


Each slack or surplus variable is assigned a price of zero. 
The problem can then be reformulated to read 


maxz = cx, Ax — b, x >0, A = (ay, jan) (4-55) 


If a slack variable is added to the inequality 7, the column of A correspond- 
ing to x4; is e;; if a surplus variable is added to the inequality 7, the 
column of A corresponding to кы is —ег 

(2) The matrix A is examined to see whether it contains an m X m iden- 
tity matrix (i.e., whether the columns ean be interchanged to yield an L, in 
A). If there is not an identity matrix, we add a sufficient number of artificial 
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vectors q: and artificial variables za; in order to obtain an identity matrix. 
We choose a very large negative value —M as the price of an artificial 
variable (we can use the same price for each artificial variable). To begin 
the simplex method, we always use an identity matrix as the basis matrix 
because, in general, it is very difficult to obtain any other basic feasible 
solution. The initial basic feasible solution is then xz = b 2 0. 

(3) Construct an initial tableau of the form given in Table 4-1. This 
tableau is easily constructed since xg = b, y; = aj. The last row giving z, 
zj — c; must be computed from 


z= саб, 2 — Cj = Свај — Cj (4-56) 


(4) The optimality criterion is then applied. 

Optimality criterion: If all г; — c; > 0, the basic feasible solution is 
optimal, $ 

When one or more zj — €; < 0, we apply Simplex Criterion I to choose 
the vector to enter the basis. 

Simplex Criterion I: Compute 


zy — cy = min (zj — cj); zj— c; < 0. (4-57) 
D 


The vector ay enters the basis. If there is a tie for the minimum value, any 
one of the tied vectors can be chosen to enter the basis. A simple rule to 
break the tie is to choose from among the tied vectors the one with the 
smallest j-index. Once ак is selected, two possibilities present themselves: 
(a) ya. € 0 for all i; "This means that there is an unbounded solution 
involving the vectors in the basis and ак. In general, an unbounded 
solution will not be basic, but there exist unbounded solutions with not 
more than m + 1 variables different from zero. 
(b) у > О for at least one 7: In this case, а new basic feasible solu- 
tion can be found having 2 2z 
(5) If at least one у 7 0, apply Simplex Criterion II to determine the 
vector to leave the basis (the column of B to be replaced by а). 
Simplex Criterion I I: Compute 
TBr in [£225 : -5 
po = min ie yk > o} (4-58) 
The vector in column r of the basis is removed and replaced by a. If there 
is a tie, any one of the tied columns ean be removed and replaced by ау. 
A convenient way to break the tie is to choose the column with the largest 
yu. M this does not break the tie, select from this last group of columns 
the one with the smallest index i. 
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(6) Compute the new tableau using either the "ring around the rosy” 
method: 


Ja = ya — Pty ади eal con EE E r (4-592) 

rk 

r=, айу, (4-59b) 
Yrk 

Хач Yo 2 = een 28—06 = уар GF = 1,..., 7, 


or the vector transformation formula: 


= Y; + yo, 

Yj = [yo Ym4i3], allj, (4-60) 
= (ea ну. ОА ДЫ! OO i 

SET [ VEN CY Uae ures Ue! ERS (АҢЫ Urk ] 


The price in the rth position of the column headed “cg” should be replaced 
by cj, and the vector in the rth position under "vectors in basis" should 
be replaced by ар. 

(7) Return to step 4. The simplex method is a finite iterative process and 
leads in a finite number of steps to an optimal (or unbounded) solution. 
There is the exceedingly rare possibility that if degeneracy appears, cy- 
cling may occur and an optimal solution will never be reached. This has 
never happened in any practical problem. Later we shall show how to 
ensure that cycling will not, take place. However, since this procedure 
involves additional computations, it is almost never used. 

In the simplex method, we know at each iteration whether the solution 
is optimal, and if not, whether we can form a new basic solution or whether 
there is an unbounded solution. In general, the number of iterations 
(changes of basis) required to reach an optimal solution lies between m. 
and 2m, where m is the number of constraints. 

As indicated in Chapter 1, the term "simplex" method has nothing to do 
with the technique as it is now used. The name "simplex" was coined in 
the early days when a geometric interpretation of the method was evolved, 
with one of the m constraints 2172; = 1. We shall briefly discuss this 
in the next chapter. 


4-11 Illustrative examples. In order to gain more familiarity with 
the actual process of solving a linear programming problem by the simplex 
method, it is desirable to work out some numerical examples. Two simple 
examples will be presented in this section. It will soon become evident that 
even fairly simple problems require considerable computational effort. 
Vor this reason, a great deal of energy has been and is being expended in 
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the development of digital computer codes for the solution of linear 
programming problems. 

Consider first the simple maximization problem solved graphically in 
Chapter 1: 


Зх + za < 15, 
5a, + 229 < 10, 


‚тт; 2 0, Fai 
max г = 52, + 312. 
After slack variables have been added, the constraints become 
32, + 522 +23 = 15, 
5z, + 222 z4 = 10. (4-62) 


Then 
a, = (3,5, a2=(,2), аз = [1,0], 
а, = (0,1), — b = [15, 10]. 
Since we added a slack variable to each constraint, an identity matrix 
appears in (4-62). Thus we immediately have an initial basic feasible 
solution with аз, a4 in the basis. The slack variables have zero prices, and 


hence for this basic solution cp = (0, 0). Furthermore, x» = [15, 10], 
z = 0,2; — cj = —cej. We can at once set up the first tableau, given in 
Table 4-2. (For clarity, we writez; — су in the last position of the “vectors 
in basis" column; this indicates that the last row is the (z; — cj)-row. Of 
course, the first element in the (z; — cj)-row, in the column headed "b," 
is 2.) 


TABLE 4-2 — TABLEAU 1 


- 
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Obviously, this initial solution is not optimal; two of the (z; — c;)-values 
are negative. The minimum value is z; — cı = —5. Therefore, a, will 
enter the basis at the next iteration. In actually carrying out the work, 
it is helpful to encircle the a;-column of Tableau 1, as shown. „Now we must 
determine the column of the basis which will be replaced by a;. Equa- 
tion (4-58) is used for this purpose. Both y1, and ys; are positive. The 
column to be removed is determined from the minimum of the following 
two numbers: { 

2 o O 

yn S рА yn 5 » 
"The second column of the basis, a4, is to be replaced by a. We encircle the 
rth row, i,e., the second row, as shown. Note that rp; ів found in the 
column headed "b" in the same row as ут and xg in the same row as Yar 

Now it is necessary to compute the tableau for the new basic feasible 
solution. Suppose that we use the “ring around the rosy” method. In this 
ease, the rth row, i.e., the second row beneath the headings, is the special 
row; we wish to find it first. According to (4-59b), the new rth row is 
obtained by dividing each element of the old rth row by уш = yor = 5. 
For example, 


1 
бю = ie 00 Daa, йз = 2 = 04, 04 = = = 0.2. 


ya 5 
"The remaining two rows are filled by means of (4-592). For row 1, 


3. 
Yk уд 5 98. 


ou 


Therefore, 
fij = уу — 0.6y2;. 
In particular, 
tio = 15 — (0.6)(10) = 9, fi» = 5 — (0.6)(2) = 3.8, 
f = 0 = (0.6)(1) = —0.6. 
For row 3 [the (z; — cj)-row], 
y: 
3 —l в faj = узу + узу; 
hence 
fo = 0+10 = 10, 54—-2—3-—,1 40+ 1= 1 


Tableau 2 is presented in Table 4-3, 
_ The new basic solution is still not optimal since 23 — cg = —]. This 
is the only n — Cj. Therefore, az will be inserted at the next 
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TABLE 4-3.— TABLEAU 2 


ej 


Vectors 


step. The column of the basis to be removed is determined from the 
minimum of 


(The values are, of course, read from Tableau 2 since it contains the current © 
basic feasible solution.) Thus column 1 of the basis, аз, is replaced by аз. 
Another tableau is now computed. This time the rth row is row 1, and 
we compute it first. ‘The new row 1 is found from row 1 of Tableau 2 by 
dividing each element of Tableau 2, row 1, by yr = yia = 3.8. Hence 


q 1 
йо = эв = 2.368; fis = 387 0.2632; 


Pa eng 08 = —0.1579. 


Rows 2 and 3 are found by means of (4-59a). For row 2, 


ym _ J22 94 _ 94053 and зу = yaj — 01059» 
Yrk — Vi2 3.8 


In particular, ; 
fap = 2 — 0.1053(9) = 1.053; das = 0 — 0.1053(1) = —0.1053; 
fos = 0.2000 + 0.1053(0.6) = 0.2632. 


For row 3, 


yas... 1 _ —02032; з = Yas + 0.2632y1,; 
yi2 3.8 
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TABLE 4-4 — TABLEAU 3 


CB 
3 0.2632 | —0.1579 
5 


—0.1053 0.2032 


0.2632 | 08421 | 


hence 
зо = 10 + 0.26329) = 12.37; — 35 = 0 + 0.2632(1) = 0.2632; 
031 = 1 + 0.2632(—0.6) = 0.8421. 


Tableau 3 is given in Table 4—4. 

In Tableau 3, 2; — cy > 0 for each aj. Consequently, we have obtained 
an optimal solution. The optimal basic solution has aj, a in the basis, 
From column b, we see that zı = 1.053, х = 2.368, г = 12.37. Thisis 
precisely the answer that was obtained in Chapter 1. We can actually fol- 
low each iteration on the diagram of Fig. 1-1, For Tableau lizi = х3 = 0, 
and we are at the origin of Fig. 1-1. In Tableau 2, 2, = 2, za = 0, and 
we have moved to the extreme point on the z;-axis. Finally, in Tableau 3, 
we move to the optimal extreme point. 

As a second example, let us solve the problem 


Ti + 32 + 23 + 5z4 < 20, 
271 + 10z; + 23 + z4 > 4, 
321 — 43 — Šta + 102; < —10, 


71, 22, 13, Z4. > 0, 


(4-63) 


min z = —2z, — ту — 4r — B. 


To.ilustrate the procedure, we shall convert this to a maximization 
problem by changing the signs of the Prices, i.e, we want to maximize 


2 = 211 + xg + 4x + 514, with minz = —max ғ. 
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The constraints must now be considered. We wish to have b> оп 
order to obtain an initial basic feasible solution. Hence the third inequality 
must be multiplied by —1. This gives 


—32, + хә + 5х3 — 10r, > 10. 
After adding slack and surplus variables, we obtain 
жу + 3x2 + 223 + 524 + 25 = 20, 
2z, + 1625 + za + t4 — 26 = 4, 
—32, + 22 + 52g — 10r, — 27 = 10. 


Clearly, an identity matrix is not present. We do have one column e, (for 
slack variable ть) of an identity matrix. We then add two columns, 
qı = es, q2 = es, and two artificial variables a1, za». Prices —M are 
assigned to Tai, Za2- 

We can now construct the first tableau, shown in Table 4-5. The initial 
basis matrix is B = I = (ag, qı, do), 80 that cg = (0, —M, —M). For 
this tableau, 

zj — Cj = Свај — Cj; з = cpb; 


thus, for example, 
nd M SA 4 0M 4, 
= —4M — 10M = —14M. 


It.will be noted that the expressions for 2, z; — cj contain two terms of 
widely different magnitudes, that is, 2, 2; — c; are of the form aM + 8. 
Since M is a very large number, 8 can be neglected with respect to aM. 
However, if the 8-terms are dropped completely, we shall run into difficulty 
when the last artificial vector is removed from the basis. Then the aM- 
terms go to zero (because there are no more artificial vectors in the basis), 
and only the -terms will be left. If the 6-terms аге dropped, the trans- 
formation formulas cannot be used to compute the new 2, 23 — су atthe 
iteration at which the last artificial vector is removed. Instead we must 
apply the defining equations 2 = CBXB, £j — cj = cay; — cj. И the 
B-terms are retained, as in this example, the transformation formulas can 
always be'used. Furthermore, if there are tied vectors, it is possible to use 
the B-terms to determine the vector to enter the basis. When the aM- 
terms are the same for two or more vectors, one can attempt to break the 
tie by examining the 8-terms. ^ 

<0. The smallest is 22 — С; = 


Two vectors az, az have 23 — % S. 
—17M — 1. Therefore a2 enters the basis. All yia > 0; hence the column 
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of the basis to be replaced by az is the minimum of 


ag SBR вало 
уз 3’ yn 16° уз; 1 
"The smallest value is z52/y22, and column 2 of the basis is replaced by аз, 
that із, qı is removed. We then encircle the column of the vector to be 
inserted and the row corresponding to the column to be removed, as 
shown. 

Now the new tableau must be found. In the present example, we shall 
illustrate the use of the vector transformation formula (4-60). First we 
compute Ф. Since k = 2, == 2, yy = yoo = 16, and 


31] 1 1M +1 
o=|- 34-14-42 16 | 


= [—0.1875, —0.9375, —0.0625, 1.0625M +- 0.0625]. 
After finding «b, it is easy to determine all the new columns. For example, 
for Yo, we have yro = yao = тв; = 4, and 
€ = (20, 4, 10, —14M] 
+ 4[— 0.1875, —0.9375, —0.0625, 1.0625M + 0.0625] 
= [19.25, 0.2500, 9.750, —9.750M + 0.2500]. 

"This is indeed the "b"-column of the new tableau (Table 4-6). The other 
columns are found in precisely the same way. Note that the same Ф is 
used in the determination of each new column. 

At the next iteration, we see from Tableau 2 that аз will enter the basis, 
and column 3, that is qa, will be removed. In this step, we remove the last 
artificial vector, and hence in Tableau 3, the M-terms disappear in ? and 


pins су. For this iteration k = 3 and r = 3, so that y, = Vas = 4.938, 
ar 


PT [- 1.813, 0.06250, 1. — | 4938M + 3.038 
4.938 0 


1938' 4498 '4938 — 
= [-0.3671, —0.01206, —0.7975, M + 0.7974]. 


To see how the M drop out, suppose that we compute $4. Since ya = 
Yas = —10.06, y 


Ӯ, = [4.812, 0.00250, —10.06, 1000M — 4.938] — 
10.06{—0.3671, —0.01266, —0.7975, M + 0.7974] 
= [8.506, 0.1899, —2.038, — 12.96], 
which is the column given in Tableau 3. 
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As has been indicated previously, it is in general unnecessary to include 
columns in the tableaux for the artificial vectors. In the present example, 
the artificial vectors are transformed at each iteration for later use in the 
problems. 

It is unnecessary to discuss in detail the arithmetic involved in com- 
puting the remaining tableaux (Tables 4-8, 4-9, 4-10). 

Note that, in this case, the simplex method was not too efficient. To 
obtain an optimal basis involving only three vectors, five iterations were 
required—two more than the minimum if all the correct vectors had been 
inserted in order. 

The maximum of Z is 40.00. Thus the minimum of z is —40.00. The 
optimal basis contains a;, аз, ав. The values of the basic variables are 
тү = 7.273, хз = 6.364, rg = 16.91. Note that one of the variables in 
the basis is a surplus variable. 


REFERENCES 


The references listed at the end of Chapter 3 apply also to this chapter. 


PROBLEMS 


4-1. Show that any vector which is removed from the basis at one iteration 
in the simplex method cannot-immediately re-enter the basis at the next iteration. 

4-2. Can a vector which is inserted at one iteration in the simplex method 
be removed immediately at the next iteration? When can this occur and when 
is it impossible? 

4-3. Recall from the discussion of the theory of the simplex method that if a; 
is to enter the basis, we consider for removal only those columns of B which have 
уњ > 0, so that we are sure fg. = zp,/y > 0. This must be done when 
there is no degeneracy. However, when degeneracy is present, a vector can be 
inserted at а zero level even if yr < 0. If the simplex method is used, why is it 
sensible to consider only the yj > 0, even in the presence of degeneracy, and 
ignore the fact that a vector might be inserted at a zero level with Urk < 0? 

4-4. In the simplex method, we consider as candidates for insertion into the 
basis only the vectors with z; — c; < 0 (for a maximization problem). How- 
ever, if a vector goes in at a zero level, it does not make апу difference whether 
2; — ¢;{|< = >}0. When degeneracy is present, does it make sense to con- 
sider only the vectors with z; — c; < 0? 

4-5. Using an example in two dimensions (two variables in the original con- 
straint inequalities), show graphically that the procedure of finding the largest 
increase in the objective function at each iteration can lead to more iterations 
than a procedure in which z may actually decrease at some point. 

4-6. Show that once an artificial vector has been removed from the basis, 
it is never necessary to re-insert it. К 

4-7. If inequalities of the form 27; a;jz; > 0 appear in a linear programming 
problem, discuss why it is desirable to convert them to the form 3^; (—a;,)z; < 0. 
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4-8. Show that if some variable appears in only one constraint of Ax — b, 

the activity veetor corresponding to that variable can be converted to a unit 
' vector. Why is it desirable to do this whenever possible? 

4-9. Prove that, by a proper choice of the prices in the objective function, 
any extreme point of the convex set of feasible solutions can be made optimal. 
In general, what is the minimum number of prices that would have to be speci- 
fied in order to make any given extreme point optimal? 

4-10. Using the results of Problem 4-9, show that the extreme points of the 
convex set of feasible solutions are connected, i.e., one can move from any one 
extreme point to any other extreme point by moving from one extreme point to 
an adjacent extreme point. When presenting the proof, ignore the problems 
created by degeneracy. x 

4-11. Show that if we are given any one tableau for the simplex method, the 
inverse of the basis matrix for the basic solution can be read from the tableau 
provided we know which columns formed the initial identity basis matrix. 
Hint: The initial tableau has the form (b, I, R). The tableau for any succeeding 
basic solution with basis matrix B is formed by multiplying the initial tableau 
by B71, 

4-12. Suppose that, we are given some tableau for the simplex method. Is it 
possible to determine uniquely the activity vectors aj and the requirements 
vector b? Assume that the only information available is given in the tableau. 
Would it make any difference if it were known which vectors were in the original 
identity matrix? 

4-13. Solve the following linear programming problem graphically and by ' 
the simplex method. Show in the geometrical presentation the extreme point 
which corresponds to each tableau of the simplex method. 


$x, + 519 < 15, 621 + 222 < 24, 21,22 > 0, maxz = 201 + 22. 
4-14, Solve the following linear programming problem by the simplex method: 
8x, + 3r + 423+ ta S 7, 
221 + 6z + 23+ 514 € 3, 
тү + 4rz + 523 + 224 € 8, 
all z; > 0, 
maxz = 321 + 4х2 + rat 724. 


4-15. Solve the following linear programming problem by the simplex method: 
тү + 522 + 9x3 — 624 > —2, 
8x1 — 22+ 13 314 < 10, 
ori ~ 3r2 + Тїз — 834 > 0, 
all z; > 0, 


maxz = 2x; — 312 + 4r3 + T4. 
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4-16. Solve the following problem by the simplex method without adding any 
artificial variables: 


Зір + 212 + 423+ 24+ 525 = 15, 
zı + 223 + z3 + 524+ 525 = 13, 
2х3 + 614 325 > 6, 

allz; > 0, 
maxz = zı + 312 4- 523+ 424+ 9х5. 


4-17. Solve the following linear programming problem graphically and by the 
simplex method, using Charnes’ — technique for the artificial vector. Show 
the correspondence between the simplex iterations after a feasible solution has 
been found, and the extreme points of the convex set of feasible solutions. 


821+ 47256, 2113122 2, 2,222 0, maxz = xı + бло. 


4-18. Using Charnes' —M technique to handle the artificial variables, solve 
the following linear programming problem by the simplex method: 


бі — 2r2 + хз — 304 > 2, 

6zı + z2 — бхз — 314 > 5, 

—11 + 422+ 3z3 + 724 > 6, 
айт; > 0, 

minz = 3zi + 4r2 + 23 + 624. 


4-19. Using the —M technique to handle the artificial variables, solve the 
following linear programming problem by the simplex method. What can be 
said about the optimal solution? 


201 + 8r2— 523+ 424 = 7, 
—8х1+ 272+  3z3— 924 = 8, 
1.7511 — 22+ 3.2513 — 4.2514 = —4.75, 

all z; > 0, 
maxz = 82; + х2 + 3x3 + 5z4. 


у 4-20. Using the — M technique for the artificial variables, solve the following 
linear programming problem by the Simplex method. What can be said about 
the optimal solution? 


4zi + öz Û лз — 3x4 = 5, 
211 — 322 — 413 4- 524 = 7, 
Tı + 412 + 2.53 — 4z4 = 6, 
allz; > 0, 
maxz = 3z1+ 2z2+ 23 + 474. 
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4-21. Consider the linear programming problem Ax — b, x 2 0, maxz — cx. 
Suppose that A is a matrix with many rows and columns. However, assume 
that it can be written in partitioned form as 


Ai 0 e 0 
0 А»... 0 


O Ж А I9 "М 


Show how such a problem сап be simplified by breaking it down into a series 
of smaller problems, 

4-22. In the simplex method, the eriterion determining the vector to enter 
the basis selects the vector which gives the greatest unit increase in 2 (i.e., the 
greatest increase in z if one unit of z, was inserted). Consider instead the follow- 
ing criterion: We wish to select the vector which maximizes c(t — x), where 
£is the new n-component vector for Ax = b. What is the geometrical interpre- 
tation of this criterion? What does it reduce to analytically? Is this criterion 
identical with any other criterion considered so far? 

4-23. Suppose that in solving some linear programming problem, we reach 
a stage where one or more artificial vectors are in the basis and an indication 
of an unbounded solution is obtained (i.e. for the vector ax to enter the basis, 
all yix < 0). Discuss the various possible meanings of such a situation. Show 
that by Charnes’ —Af method feasibility will be reached before an indication 
of an unbounded solution appears, if there is a feasible solution. 

4-24. In this chapter, preliminary transformations were always made to 
ensure that b > Oin Ax = b. Show that these are really unnecessary. Demon- 
strate that with b; < 0, we can get started if the ith column of the initial basis 
matrix is —e; instead of e; Thus if some of the b; are negative, the original basis 
matrix can contain eolumns of the form —e; as well as columns of the form e;. 
What is the inverse of a matrix which is an identity matrix except for the fact 
that some of the diagonal elements are —1 instead of 1? How does one obtain 
the first tableau? Show that the number of artificial vectors needed will not 
depend on whether or not a transformation is made to make the b; non-negative. 

4-25. When making simplex computations by hand, it is very important to 
have some means for making numerical checks on the computations at each 
iteration. Compute the number of operations (multiplications and divisions) 
required to make the following checks: 

(a) Substitute the basic solution into the set of equations Ax — b to sce 
whether the solution satisfies the equations. 

(b) Compute the z; — c; from 2; — ej = Cys — CH and check to see 
whether the same result was obtained by the transformation formulas, 

(c) Find B-! in the eurrent tableau and multiply it by B (found in the 
initial tableau) to see whether an identity matrix has been obtained. 

(d) Compute the y; from y; — B-!aj and see whether the result is the 
same as that given by the transformation formulas. 
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4-26. Find B-! in Table 4-7. Compute B~!B. Is the resulting matrix an 
identity matrix? 

4-97. In Table 4-8, substitute the basic solution into the original set of 
equations. Does this "solution" satisfy the equations? 

4-28. A very convenient way of performing a numerieal check on the computa- 
tions at each stage of the simplex method is to use a so-called sum-check column. 
This is done as follows: Define the vector o as 


^ . 
o=b+) a+} a 
j=l ici 
and the vector 2 as 
nds 
Q9 = B's = xj Уу; = lu... ul. 


j=l 


For a given tableau, 2; can be obtained by adding all the elements in row + of the 
tableau. However, 2 also obeys the same transformation relations as all other 
columns of the tableau, i.e., 


бош Os. ol ELEM m 
Yrk 


Prove this. Suppose that we annex the vector 2 = с as an additional column 
of the initial tableau. When transforming the tableau, we then transform 
2 = ø, using the usual transformation formulas. In the new tableau we add all 
elements in row û (except in column ©). This should yield the element О; in the 
f-column. Add a sum-check column to each of the tableaux in Tables 4-5 
through 4-10. Transform this column and then perform the appropriate sum- 
mations to show that a check has been obtained. Show that the sum check can 
also be used on the (2; — cj)-row. Note that the q; need not be included in є 
unless the artificial vectors are included in the tableaux and are transformed at 
each iteration. і 

4-29. How many additional operations (multiplications and divisions) are 
required at each iteration in order to carry along a sum-check column? 

4-30. Make a list of the type of errors which can be made during simplex 
calculations. Are any of these self-correcting? Which errors will lead to an in- 
correct answer? 


CHAPTER 5 
FURTHER DISCUSSION OF THE SIMPLEX METHOD 


"Once more unto the breach, dear 
friends, once more." 
Shakespeare, Henry V. 


5-1 The two-phase method for artificial variables. In the preceding 
chapter, we have made the following observations: It is frequently nec- 
essary to add artificial variables and vectors to the set of constraints in 
order to find easily an initial basic feasible solution to a linear programming 
problem. If the original problem is to be solved, the artificial variables 
must be driven to zero. This can be achieved by assigning a large negative 
price (— М) to each artificial variable, thus making it unprofitable to have 
such variables in the basis. 

There are several aspects of the —M method which can lead us into 
difficulty, especially if the problem is to be solved on a digital computer. 
To use a digital computer, M must be assigned some numerical value 
which is much larger than any of the other prices appearing in the problem. 
Since a computer has only a fixed number of digits, the large difference in 
order of magnitude between M and the other prices can cause problems. 
We have already noted that the z; — c; will be of the form aM + 8 when 
artificial vectors are in the basis. If the computer, for example, has only 
four digits, and if aM = 1014, 8 = 1.472, then to four figures, aM + 8 = 
1015; this implies that three significant figures in 8 have been lost. How- 
ever, when the last artificial vector is driven from the basis, the aM-terms 
drop out, and only the -terms remain. If most of the significance in the 
B-terms has been lost, then z, z; — с; will need to be recomputed from their 
definitions. Consequently, in the computer code a special procedure will 
often be required for the iteration at which the last artificial vector is 
removed. Of course, at every iteration, we could compute the 2; — c; 
from the definition, but this would again result in more work than is really 
needed. ^ 

There is a further difficulty: If M is not chosen large enough, an incor- 
rect answer may be obtained. For example, if a is very small, it may turn 
out that 8 is not negligible with respect to «M. Hence one might conclude 
that there is no solution when in reality a solution exists. 

Another method of treating artificial vectors has been developed by 
Dantzig, Orden, and others at the RAND Corporation. Although they 
were searching mainly for a technique that would be suitable for use on 
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digital computers, they developed a method, called the “two-phase tech- 
nique," which can be used equally well for hand computations. In this 
chapter we shall present a form of the two-phase technique which is useful 
for hand computations, while Chapter 7 will discuss a different format 
which can also be used for hand computations, but which is espectally 
well suited for use with digital computers. 

The general idea of the two-phase method is as follows: Suppose that we 
try to solve the linear programming problem in two parts. First we 
drive all artificial variables to zero. This is called Phase I. Then we 
maximize the actual objective function z, starting from a basic feasible 
solution which either contains no artificial vectors or some artificial 
vectors at a zero level, This is Phase II. 


5-2 Phase I. In Phase I, let us assign to an artificial variable a price 
of —1 rather than a price of —M. To all other variables we assign the 
price 0 (regardless of what their prices are in the actual problem). Then, 
instead of considering the actual objective function, we maximize the 
function 


C» Nee а te 6D 
izi 

where xa is an s-component column vector containing the artificial variables, 
It will be recalled that the initial basic feasible solution to the augmented 
problem has xa > 0. Furthermore, the simplex method prevents any 
variable from ever becoming negative. Consequently, the artificial varia- 
bles will always be non-negative. Hence it is always true that z* < 0, 
and the maximum value of z* is zero; z* will be zero only if each artificial 
variable is zero. 

If max z* = 0, we drive all the artificial variables to zero by maximiz- 
ingz*. If max z* < 0, then the artificial variables cannot be driven to zero, 
and the original problem has no feasible solution, Thus Phase I consists of 
the usual simplex calculations except for the fact that we are maximizing z* 
rather than z. The price of each legitimate variable is set to 0, and a price 
of —1 is assigned to each artificial variable. 

In Phase I, we stop as soon as z* becomes zero, because we know that 
this is the maximum value of z*. We need not continue until the optimality 
criterion is satisfied, if z* becomes zero before this happens. 

During Phase I, the sequence of vectors to enter and leave the basis is 
the same as in Charnes’ —M method. It is not hard to see why this is во, 
With the —M method, 

2 — c; DDE f (5-2) 
r 


for any legitimate vector aj. The sum is taken over those r that correspond 
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to the columns of the basis containing artificial vectors; 8 contains the 
terms involving the ordinary prices, and is assumed to be negligible with 
respect to the —M term. However, in Phase I, 


= 5 (5-3) 


for any legitimate vector because the price- of an artificial variable is —1 
and that of a legitimate variable is 0. Since 8 is negligible, this expression 
differs from the гу — су of the —M method only by the factor M. Thus in 
both cases, the same vector will be chosen to enter the basis. 

The only possible exception is the situation where the vectors are tied. 
In this instance, the 6-terms tan be used to break the tie in Charnes’ 
method. This 8-term does not appear in the two-phase method, and there- 
fore some other procedure must be used to break the tie. In such a case, 
the vector entering the basis in Charnes’ method might be different from 
that in the two-phase method. 
At the end of Phase I (when the ‘optimality condition is satisfied or 
z* = 0), three possibilities exist: | А 


(1) max 2* < 0; one or more artificial vectors appear іп the basis 
at a positive level. The original problem has no feasible solution. 

(2) max 2* = 0; no artificial veetors appear in the basis, We have, 
found a basic feasible solution to the original problem. 

(3) max #* = 0; one or more artificial vectors appear in the basis at 
a zero level. We have found a feasible solution to the original problem, 
Because some artificial vectors appear in the basis at a zero level, there 
may be redundancy in the original constraint equations, 


5-3 Phase II. When Phase I results in (2) or (3), we go on to Phase II 
to find an optimal solution, In Phase I, we assign ils actual price ¢, to 
each legitimate variable and a price of 0 to any artificial variables which 
may appear in the basis at а zero level. The function to be optimized in 
Phase TI is the actual objective function z. The first tableau for Phase 11 
is the last tableau for Phase I, the only difference being that the (z; — cj)- 
row for the last tableau of Phase I must be altered to take account of the 
change in the prices. The new (2; — cj)-row is found from ғ = сихи, 
z; — су = сву; — су. The last tableau of Phase I can be used as the 
first tableau of Phase IT because it gives a feasible solution to the original 
set of constraints, 

When Phase I ends in (2), no artificial variables appear in the basis. 
Therefore we start off Phase IT with a basic feasible solution to the original 
constraints, and there are no problems. If, however, Phase I ends in (3), 
we must give some attention to the artificial variables which appear in the 
basis at a zero level, We must make sure that the artificial variables never 
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become positive in Phase II. A priori, we cannot be certain that the 
artificial variables will remain zero because they have a price of zero in 
Phase II. In Charnes method this problem does not arise: Once the 
artificial variables go to zero, they will remain zero because of the price 
—M ; if they became positive, z would be worsened, and by the nature of 
the simplex method this cannot happen. 

In the section which dealt with redundancy, we saw that if we had a 
basic feasible solution with artificial vectors in the basis at a zero level, 
we could either remove these artificial vectors and obtain a degenerate 
basic feasible solution to the original problem; or if the artificial vectors 
could not be removed, the redundant equations could be eliminated, and 
we would obtain a basic feasible solution to the new set of constraints with 
the redundant equations removed. Hence, if Phase I ended in (3), we 
could, theoretically, remove all artificial vectors which appear at a zero 
level. However, the insertion of legitimate vectors into the basis and the 
removal of the artificial vectors would not necessarily bring us any closer 
to an optimal solution. This method could therefore involve a great deal 
of unnecessary work, and for this reason, we do not attempt to use it. 
If yi; = 0 for all j and for those i that correspond to the columns of the 
basis containing artificial vectors, we shall show below that we can cross 
off the rows with all y;; = 0 and drop the corresponding artificial vectors. 
This allows us to proceed with a smaller basis and tableau. 

Let us now study how to proceed in Phase II when at the beginning one 
or more artificial variables appear at a zero level and for at least опе û 
corresponding to a column of the basis containing an artificial vector, 
there exists one or more y; з 0. We must make sure the artificial variables 
remain zero at each iteration in Phase II. Suppose that at some iteration 
in Phase II we insert the vector a; into the basis. If Yik > 0 for one or 
more 7 corresponding to columns of the basis containing artificial vectors 
at a zero level, then our criterion determining the vector to be removed 
indicates that an artificial vector can be removed. Then a, enters the basis 
at a zero level. This means that the values of the variables in the basis 
remain unchanged, and therefore any artificial variables in the new 
basis remain zero. No difficulty can arise in this case. 

Next imagine that a; is to enter the basis, and yi = 0 for all i corre- 
sponding to the columns of the basis containing: artificial vectors. No 
artificial vector will be removed. Assume that the legitimate vector in 
column r of the basis is removed. The values of the artificial variables in 
the new basic solution are 


£pi = їв; — VA Ep, =0 (5-4) 


r 


because xg; = 0 and y = 0. This possibility causes no difficulty either. 
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At this point, it is instructive to note one more thing. Consider the sit- 
uation where y;; = 0 for all legitimate vectors when 7 corresponds to a 
column of B containing an artificial vector. The values of ĝi; are given by 

Di = yg — yy 0. (Mj) (5-5) 

Urk 

because yi; = Yik = 0. The new tableau will again have a row of zeros. 
This demonstrates that the artificial vector in column Т of B will never be 
removed. For this reason, we can cross off the row that will always contain 
zeros and drop column î of the basis. We then continue the computation 
with a new tableau (reduced by one row) and an (m — 1) X (m — 1) 
basis. The above result shows why, at the end of Phase I, we can cross off 
rows with y;; = 0 for all j when 7 corresponds to an artificial vector. 
Incidentally, this applies not only to the two-phase method but also to 
Charnes’ method. It is merely another way of stating the result obtained in 
Section 4-6 which says that if the above conditions hold, constraint 7 is 
redundant and can be dropped. 

In the last case to be examined, the a; to enter the basis has yj, < 0 
for all i that correspond to the columns of B containing artificial vectors, 
and ya, < 0 for at least one i. In this instance, the usual criterion which is 
used to determine the vector to leave the basis would not consider any 
artificial vector for removal. A legitimate vector would be removed 
instead. If a legitimate vector b, is removed and xg, > 0, the values of 
the artificial variables with у < 0 in the new basis are 


Vik Yik 
= a ee = — > 0. 5-6 
fpi = Thi "e Tar "m Tar (5-6) 


Here we have encountered the only possible cireumstance under which the 
artificial variables in the basis can become positive; that is, ук < 0 for û 
corresponding to one or more of the columns of B that contain artificial 
variables. It is now also clear how to avoid this possibility. Instead of 
removing a legitimate vector, we remove any one of the artificial vectors 
with yi < 0. Because у # 0, we are sure that the new solution will 
be basic. It will also be feasible because the artificial vector was at.a zero 
level, and therefore a; will enter at a zero level. Furthermore, 2 — z. 

Let us summarize what we do in Phase II when artificial variables appear 
at a zero level in the basis at the end of Phase I. First, if at the end of 
Phase I, y; = 0 for all j and for û corresponding to a column of B that 
contains an artificial vector, this row (or rows) is crossed off the tableau, 
and we begin Phase II with a basis of smaller size. This can be done 
because we have shown that the corresponding artificial vector will never 
be removed, and hence that there js redundancy in the constraints. 
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In Phase II, we apply the usual criterion to determine the vector to 
enter the basis. We also apply the usual criterion to determine the vector 
to leave the basis unless у < О when 7 corresponds to a column of B 
containing an artificial vector and y; < 0 for at least one such 7. Then, 
instead of removing some legitimate vector as the usual simplex criterion 
would require, we remove one of the artificial vectors with yi, < 0. In 
this way, we ensure that the artificial vectors will never become positive. 

If, in Phase II, artificial vectors appear in the basis at a zero level, the 
sequence of vectors entering the basis will, in general, be different from that 
obtained by the —M method. Because of the — M, the artificial vectors 
would control the vector to enter the basis (except where y;; = 0 for all j 
and for corresponding to the columns of B that contain artificial vectors, 
in which case these rows should be crossed off). In Phase II, the price of 
an artificial vector is 0 and does not affect the vector to enter the basis. 
‘The two-phase method would seem to be preferable in this case, because it 
selects the vector to enter the.basis as if no artificial vectors were present. 


5-4 Numerical examples of the two-phase method. It will be helpful 
to illustrate the two-phase method by considering two simple numerical 
examples. As the first example, consider the problem worked out by the 
—M method in Chapter 4 (Tables 4-5 through 4-10). In the two-phase 
method, we begin by assigning prices of zero to the legitimnte variables 
and of —1 to the artificial variables. For the problem at hand, we desire 
to maximize 2* = —2a1 — Taz in Phase I. We shall denote the 2; — с, 
and cg of Phase I by 2 — cj and ch. The first tableau, constructed in the 
usual way, is given in Table 5-1. Using the prices appropriate to Phase I, 
we compute the z¥ — cf. In the first step, a» enters the basis. As expected, 
аз and the vector which first entered the basis with the —M method are 
identical. Tables 5-2 and 5-3 contain the remaining tableaux of Phase I. 
At the end of Phase I, there are no artificial variables in the basis. We 
assign to the legitimate vectors their actual prices c; and recompute the 
(г; — cj)-row. This yields Table 4—7. We then proceed with Phase II and 
obtain Tables 4-8 through 4-10. 

As a second example, let us solve the following linear programming 
problem: 

32, 4-223 H 3+ 424 < 6, 


2a, + 2 + 5х3 + 24 € 4, 
221 + 6x2 — 4x3 + 8274 = 0, 
тү + 3x2 — 203 + 4r4 = 0, 
Ti, 22, 23, 34 > 9, 

maxz = 2, + 1.522 + 523 + 224. 
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TABLE 5-1 — TABLEAU 1: Puase I 


et о1о [0] 0/0] 0] 0)-1;/-1 
Vectors 
ch in b | a: | az | аз | ag |аѕ | ав | ar | qı | Q2 
Basis 

0 аз 20 1 3 2 5|1 0 0| 0 0 
=l qı 4 2 16 1 EC ESI 0| 1 0 
x ч? 10 | —3 1 5|—10, 0 0| —1| 0 1 
21 = сў —14 PREL 6 9/0 1 1| 0 0 


After the addition of slack variables the constraints become: 


Bx; + 2159 + 23 + 4x4 + Ts = 6, 
2x, + te + 5ta + t4 +26 = 4, 
2r, + Gre — 413 + 84 = 10, 
х + 3z5 > 213 + 4x4 Ex. 


Since the last equation is one-half the third one, there is clearly redundancy 
in the constraints. Rather than dropping the redundant constraint, we 
shall retain it to show what happens in the two-phase method. 

Two artificial variables are needed to obtain an initial basie feasible 
solution. Because of the redundancy, we know that one artificial veetor 
will be in the basis at the termination of Phase II. This example then 
gives us the opportunity of demonstrating how to handle artificial variables 
in Phase IL. Note that in the initial basic feasible solution both artificial 
variables are equal to zero. Thus, Phase I is not required, and we begin 
with Phase II. The tableaux for Phase II are given in Tables 5—4 through 
5-6. 

At the first iteration, the usual rule for determining the vector to leave 
the basis would yield ag. However, if this were done, both artificial vari- 
ables would be positive at the next stage. Hence we must use our alterna- 
tive rule and remove one of the artificial vectors. It was arbitrarily decided 
to remove qı. In the second tableau, the row opposite qz contains only 
zeros. Thus we know that q» will always be in the basis. Therefore, we 
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I ї 0 0 0 0 0 0 
202'0 99610'0— €6060— | 992100 0 860с— I 0 
99c10:0— | 658900 992100 6c£900— | 0 668г'0 0 I 
1!9&0— | 9roro— 1298'0 9r9ro I 90€'8 0 0 
zb 1b 15 9e Sg re tg св 
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TABLE 5-4 — TABLEAU 1 


с; 1 1.5 5 2 0 0 
Vectors 
св in b а a2 as 84 a5 a6 
Basis 
0 as 6 3 2 1 4 1 0 
0 a6 + 2 1 5 1 0 1 
0 qi 0 2 6 —4 8 0 0 
0 q2 0 1 3 —2 4 0 0 
2j — 6j 0 —1 —158 | —5 | —2 0 0 


TABLE 5-5 — TABLEAU 2 
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TABLE 5-6 — TABLEAU 3 


cross off this row and proceed with a tableau containing one row less than 
the preceding one. The next tableau gives the optimal solution. The 
appearance of q» in the optimal solution verifies that the last constraint 
equation is not linearly independent of the others. If it had been dropped 
at the start, we would have obtained the same tableau as in Table 5-6. 
No columns for artificial veetors were included in the tableaux. 


5-5 Requirements space. There are two multidimensional euclidean 
spaces which are important in the solution of a linear programming 
problem. First, there is the n-dimensional space which contains the solu- 
tions x to Ax = b. This space is called the “solutions space." Then there 
is an m-dimensional space, elements of which are the activity vectors aj 
and the requirements vector b. This space is called the “requirements 
space.” In this section, we shall present a discussion of the requirements 
space. 

Let us begin by summarizing the results of Chapter 2. The set of ac- 
tivity vectors a;,..., An can be imagined to generate a convex polyhedral 
cone C in the requirements space. This cone will be m-dimensional since 
r(A) —'m (or the rank of the matrix including the artificial vectors is m). 
We saw that a feasible solution to Ax = b will exist if and only if b is an 
element of the polyhedral cone generated by the aj. 

{A slightly different interpretation can be given to this necessary and 
sufficient condition for the existence of a feasible solution, i.e., the a; can 
be thought of as generating a convex polyhedron P also. This polyhedron 
is the set of points Ap for all м such that Iu = 1, и > 0. Suppose that 
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Figure 5-1 Figure 5-2 


there is à feasible solution x to Ax = b, Divide by Іх and write м = x/1x. 
Then lg = 1, д > 0, and Ap = b/Ix. Now b/Ix is a point on the half- 
line generated by b. "Therefore we can say that there will be a feasible 
solution to Ax = b if and only if at least one point on the half-line gen- 
erated by b is an element of the convex polyhedron generated by the ‘a; 
(see Fig. 5-1).] 

Given any set of m linearly independent vectors from A, these vectors 
also generate a convex polyhedral cone C y of dimension m. Furthermore, 
Cn CC. If the vectors which generate Cp yield а feasible solution to 
Ax = b, then b € Cg. Now consider what is done in the simplex method, 
The basis vectors span an m-dimensional cone Cp: At the next iteration, 
one of the vectors is removed and replaced by another vector. This gives 
а new m-dimensional polyhedral cone Сй. However, b є Ch because the 
new basic solution is feasible, ‘Thus in the simplex method, we move from 
one m-dimensional polyhedral cone in the requirements space to another 
in such a way that b is an element of each of these cones, 

Let us examine a specific example to get a geometric feeling for what 
happens in the simplex method. Suppose that the constraint equations are 


xay =, (-7) 


jml 


where the vectors aj, b are plotted in Fig. 5-2. 

In the simplex method, we always start out with an identity matrix as 
the basis, and therefore the initial basis of our example will contain a), аз. 
The convex cone spanned by these vectors is the non-negative orthant. 
Since b > 0, this initial basis will always give a feasible solution, that is, 
b will always be an element of the cone spanned by the unit vectors. 
Assume that, at the first iteration, a is inserted into the basis. We see 
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Аа 


а 


аз 
Figure 5-3 


that we cannot remove a; while still maintaining a feasible solution. 
Thus а; must be removed. The polyhedral cone spanned by aj, ав does 
contain b, and hence the new solution will be feasible. If aş is inserted next, 
a, must be removed because the cone spanned by a;, as does not contain b. 
However, b is in the cone spanned by as, as, etc. 

It will be recalled that the solution to a linear programming problem is 
unbounded if for some vector aj, z; — с; < 0, and y;; < 0 for all i. This 
can be illustrated geometrically in the requirements space, as shown in 
Fig. 5-3. Note that аз can be written 


аз = —aj8, — оза», 01, a > 0. 
However, b can be written 
1,8; + та; = b, 11,29 > 0. 
Adding and subtracting баз, we find 
(к + 601); + (za + ваз)аз + баз = b, 


and the solution will be feasible for any 0 > 0. If 23 — сз < 0, then z 
can be made arbitrarily large, and an unbounded solution exists. 

The idea of the requirements space can be generalized somewhat to yield 
a more complete geometric interpretation of the simplex method. To each 
vector a; and b, we shall add an (m + 1)-component. These new vectors 
will be denoted by аў and b*; they are defined by 


aj = [ае]; b* = [bal (5-8) 
We note that for any feasible solution to Ax = b, 
Уга; b* (5-9) 


Now we can think of the аў as generating a convex polyhedral cone in a 
space of dimension m + 1, which we shall call the “extended requirements 
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space.” Consider a set of m basis vectors bı, . . . , bm which yields a feasible 
solution to Ax — b. We can define 


bi = [bs end; (5-10) 
then 


c * * 
dS cab; = b”. 
i=l 
Also note that 
la; 2j] = Xwibi- (5-11) 


It should be observed that [a;, с] will not be a linear combination of the bf 
unless 2; — с; = 0. Thus if af is not one of the bř and 2; — cj # 0, 
a* will be linearly independent of the Ъ? in the extended requirements space. 

In the simplex method, we are searching for a set of m, b? which will 
yield a feasible solution and maximize the value of the last component of 
b*. The basis vectors bf generate a polyhedral cone of dimension m in 
E”*1) and therefore they lie on a hyperplane passing through the origin. 
The point b* lies on a line through b which is parallel to the z-axis. The 
intersection of this line with the hyperplane spanned by the b? determines 
the actual b* for the given basic feasible solution, that is, it determines 2. 
If we have a set of b; such that the b; yield a feasible solution, and if there 
is an a, with гь — cy < 0, then af is linearly independent of the bř, and 
the m + 1 vectors, b ( = 1,...,m) and a£ span an (m + 1)-dimensional 
cone in the extended requirements space. Furthermore, the m-dimensional 
cone spanned by the bf is a face of the (m +- 1)-dimensional cone spanned by 
the b* (i = 1,..., m) and af. This follows immediately because if 
vb! = 0 (i = 1,... m), then the proper choice of v will yield va? > 0 
since až is linearly independent of the by. Any point w in the cone spanned 
by the b? and a£ satisfies vw > О since w can be written w = Xa МЫ + 
Малай, MN 2 OG = 1,...,т-++ 1). Thus the hyperplane spanned by 
the b is an extreme supporting hyperplane for the cone spanned by the 
bř and aj. ; 

Because of (5-11) the vector [ax, 2x) lies in the hyperplane spanned by 
the b. Intuitively, it is merely the projection of az parallel to the c;,-axis 
onto this hyperplane. When a; replaces b, to yield a new basis, then the 
new set of b? also generates an m-dimensional polyhedral cone which is a 
face of the cone spanned by the original set of b? and ау. In general, how- 
ever, the intersection of the line through b parallel to the z-axis with the 
new face will be higher up the z-axis than the intersection of the same line 
with the preceding face. 

The above argument is best illustrated by means of an example, Assume 
bı, bz compose the current basic feasible solution to some linear program- 
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Figure 5-4 


ming problem and assume that there exists a vector ay with z, — e < 0. 
The cone spanned by bf, bf, af is shown in Fig. 5-4. The vectors bf, bf 
generate one face of this cone. ‘The line through b intersects this face at 
b? and hence determines the value of the objective function z, for this basie 
solution. When a, is inserted into the basis, ba must be removed. The 
vectors bf, af also generate a face of the cone, and the intersection of the 
line through b with this face yields bf with z, > za. 

The preceding discussion is a generalization of an early geometric 
interpretation which gave the simplex method its name, If one of the m 
constraints is 377; = 1, and if this constraint is removed and treated 
separately, then the a? become m-component vectors. ‘Thus the set of all 
convex combinations of the bf (i = 1,...,m) and af forma simplex in 
E". One face of the simplex is generated by all convex combinations of the 
b. When ze — су < 0, all points in the simplex lie on or above this face. 
The line through b parallel to the z-axis intersects this face of the simplex 
in b3, with the objective function given by za. When a, replaces b,, the 
new set of b? generates а face of the simplex, and the intersection of the 
line through b with this face yields b? with z, > za. The introduction of 
convex polyhedral cones instead of polyhedra allows the removal of the 
restriction Zz; = 1 from the geometric interpretation. 


5-6 Solutions space. The convex set of feasible solutions to the con- 
straints Ax = b for the general linear programming problem lies in an 
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Frovar 5-6 


n-dimensional euclidean space which has been named the solutions space. 
We have proved that an extreme point of this convex set is а basic feasible 
solution to Ах = b, and hence at least n — т of the components of an 
extreme point are zero. The extreme points of this convex set have the 
property of lying in the interseetion of n — m hyperplanes of the form 
ex = 0 = ху. In Chapter 1, we saw that in E? and ЕЁ it is very easy to 
represent the intersection of two or more half-spaces as a convex region. 
However, it is not nearly so simple to provide a geometric representation 
in E? ог E? of the feasible solutions to the constraints when these are 
converted to a set of simultaneous equations, For example, if we start out 
with two inequalities in two variables and add a slack or surplus variable 
to each, we already have a four-dimensional space. We shall now illustrate 
the only two cases of interest which ean be exhibited in E". 

(1) The most elementary three-dimensional example is that of a single 
constraint equation in three variables, eq., 


2r, F 3ra + Arg 02, ^m Fx, > 0. 


"This equation is plotted in Fig. 5-5. The convex set of fensible solutions is 
planar, and the extreme points 1, 2, 3 lie on the coordinate axes and, as 
expected, have only one variable different from zero. 

(2) In the remaining case, the solutions lie on а line (the case of a 
unique solution is not of any interest). An example of such à. case is 


xy жа 0, 2r + Brg $30, r 20. 
After adding a slack variable, we obtain 
atte = б, Arı Fxg + хз = 30, Ti 23, 73 2 0. 
The set of solutions is the set of points lying in the intersection of these 
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Fragen 5-6 


- 


wo planes and in the non-negative orthant, The convex set of feasible 
is therefore the line AB shown in Fig. 5-6, The extreme points 
A, B. They lie in the coordinate planes and have only two nonzero 
variables. 

In the general ease, Ax = b, x > 0, r(A) = m, where Ais m X n and 
т > m. Hence we know (Chapter 2) that the solutions lie in an affine 
subepace of dimension n — m in А". (The convex set of feasible solutions 
lies in this affine subspace.) н 

Given а set of basis vectors bı, . . . , ba which yield a basic feasible solu- 
tion, and some other vector ay from A, then the set of feasible solutions to 
Ax = b of the form (see Problem 3-16) 


Ч 


Ў он = ads ть, ">o, (5-12) 


fen = min (а> Of m Ек. (5-13) 


‘This eriterion fot determining 6, is, of course, the familiar criterion used i 
the simplex method to determine the vector to be removed. If б, > 0, 
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then (5-12) is nota basic feasible solution unless @ == Pmax or 0. For 
0 < 0 € max т + 1 variables ean be different from zero. Consequently, 
the set of feasible solutions (5-12) is а line in E" which begins at the 
extreme point defined by the basis bı, ..., bq» and ends at the extreme 
point defined by the basis $,, . .., ba; by = by, (vt r by = а. 

Let us show that if бык > 0, the line defined by (5-12) which joins one 
extreme point to the extreme point corresponding to a new basic feasible 
solution in which one vector in the basis is changed is an edge of the convex 
set. ‘This in turn will demonstrate that in moving from one tableau of the 
simplex method to the next, we move from one extreme point to an adjacent 
extreme point if Omax > 0, since two extreme points are adjacent if the 
line joining them is an edge of the set (see Chapter 2). 

To prove that the set of points in (5-12) is an edge of the convex set, 
we must show that it is the intersection of the convex set of feasible solu- 
tions with a supporting hyperplane. И we assume that n — m > 1, then 
n — m — 1 of the ту will be zero for any feasible solution represented by 
(5-12). Consider the hyperplane in £", 


"S у= 0, (5-14) 


һ-=-! 


the summation being taken over the n — m — 1 variables z; mentioned 
above. Note that the basic feasible solution corresponding to bi, «> 
lies on thia hyperplane as does the feasible solution corresponding to 
6, „692 Furthermore, because the variables are non-negative, we must 
have for any feasible solution 


У #20 8-15) 


a-ni 


"Therefore the hyperplane (9-14) is indeed a supporting hyperplane to the 
convex set of feasible solutions. If none of the variables whose coefficient 
is 1 in (5-14) is >0, then the feasible solutions are given by (5-12). Each 
of these solutions lies on the hyperplane (6-14) because, in (5-14), the 
coefficient of any nonzero variable is zero. Ia feasible solution is not con- 
tained in (5-12), then one of the variables whose coefficient is 1 in (5-14) 
must be >0. However, this solution must lie in the half-space 


e» > 0, 
some 
and not on the hyperplane. Thus when @ varies between 0 and Ones, the 
intersection of the convex set of feasible solutions with the hyperplane 
(5-14) is the set of points (5-12). Hence, the line joining the two extreme 
points is.an edge of the convex set, and the two extreme points are adjacent. 
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In the event that n — m = 1, there are at most two distinct extreme 
points, and they are automatically adjacent since the line joining them is 
the entire convex set of feasible solutions. 

The above paragraphs have demonstrated that two basic feasible solu- 
tions which differ from each other only in that one basie vector has been 
changed to convert one solution to the other, are either adjacent extreme 
points or the two solutions are degenerate and are merely different repre- 
sentations of the same extreme point. Hence, when we move from one 
tableau of the simplex method to the next, we either remain at the same 
extreme point (max = 0) or move to an adjacent extreme point (Omax > 0). 

If in (5-12) y; < 0 for all û, then (5-12) is a feasible solution for any 
0 > 0. In this case, we can also show that the set of feasible solutions 
(5-12) is an edge of the convex set. The proof is merely a reproduction of 
the above development. However, the edge does not lead to another 
extreme point, because 0 can be made arbitrarily large without driving any 
variable to a negative value. The edge goes out to infinity instead. Since 
either all yi < 0 or at least one y; > 0, we see that an edge emanating 
from a given extreme point either leads to another extreme point or extends 
to infinity. 

Finally, we wish to note that the matrix A — (a1,..., an) performs a 
linear transformation on the n-dimensional solutions space and takes it 
into the m-dimensional requirements space. In particular, the convex set 
of feasible solutions in the solutions space is taken into the single point b 
in the requirements space, 


5-7 Determination of all optimal solutions. We have shown that if k 
different basic feasible solutions to a linear programming problem are 
optimal, any convex combination of these basic solutions is also an optimal 
solution. The simplex procedure, as we have discussed it, stops once an 
optimal basic feasible solution has been obtained. It is seldom that any 
effort is made to find alternative optima, In fact, most computer codes 
supply a single optimum and make no provision for determining other 
optimal basic feasible solutions (if there are any). 
information can be obtained from 
feasible solutions. Hence, 
However, there should be 
basie feasible solution to 


the activity vectors during the formulation of the probl 
The final simplex table: е 


optimal basie solutions which may exist. 
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vector can be inserted into the basis without decreasing the value of the 
objective function. 

When z; — cj = 0 for one or more a; not in the basis, any such vector 
a; can be inserted to yield a different optimal solution if y;; > 0 for at 
least one i and min (=в:/0:), yi; > 0, is positive. If a; enters at a zero 
level, we do not obtain a different solution; the result is only a different 
representation of the same degenerate extreme point. If yi; < 0 for all û, 
then a; can be inserted to give a set of optimal solutions containing at least 
two variables which can be made arbitrarily large. It is also true that if 
the optimal solution is degenerate, then any vector a; for which yi; # 0 
for any i corresponding to an в; = 0 сап be inserted into the basis, and 
a new representation of the same degenerate extreme point will be obtained. 
This can be done even if 2; — cj > 0 since a; enters at a zero level. 

The above paragraph suggests the procedure for finding all optimal 
basic feasible solutions. Starting from the final tableau, which contains an 
optimal solution to the problem, we construct a new set of tableaux, each. 
new tableau differing from the final tableau only in that one vector in the 
basis is changed. For insertion into the basis, we consider the vectors ау 
with z; — c; = 0 or vectors which can enter at a zero level (even if 
2; — cj; > 0). If a; enters at a positive level, we obtain an alternative 
optimal basic solution. When a; enters at a zero level, we do not obtain a 
different solution. However, we construct these tableaux anyway, because 
in the subsequent steps they may lead to new optimal basic solutions. 

We repeat the same procedure with each of the new tableaux, and obtain 
some other optimal solutions which may or may not be optimal solu- 
tions different from those obtained in the first step. This is continued with 
each set of new tableaux until it is no longer possible to find any optimal 
basic solutions different from those already obtained. It is desirable to 
keep a record of all optimal basic solutions to prevent the computation of 
new tableaux which only yield an optimal solution that has already been 
determined. In schematic: form, the process is represented by a treelike 
structure, as shown for a hypothetical case in Fig. 5-7. The basic solutions 
corresponding to different extreme points have different letters, and dif- 
ferent degenerate basic solutions corresponding to the same extreme point 
have different subscripts on the letters. In our example, there are six 


а а Third set of new tableaux 
--Second set of new tableaux 
سے‎ + First set of new tableaux 


E gcc oa Final simplex tableau 


FIGURE 5-7 


` 
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diferent optimal basic solutions (diferent extreme points) A, B, C, D, 
E, F. After constructing a sufficient number of tableaux, we only repeat 
solutions already obtained. 

Clearly, if a fair number of optimal solutions exists, a good deal of 
work could be involved in determining all of them, since each change of 
basis requires the computation of a whole new tableau. 

If desired, the second best and third best, ete., solutions to the problem 
сап be found. To find the second best solution, for example, compute 


min j ay =e 3—6»0, 02:0," zg: >.0, (5-10) 
"This minimum is determined for each optimal tableau. Then the minimum 
of all these minima is found in order to obtain the smallest possible de- 
crease іп 2, 

Let us now consider the problem whose final tableau is given in Table 
4-10, To the aceurncy of the computations (and this is really all that can 
ever be mid in numerical calculations where only a fixed number of digits 
is retained) there is à vector ay (not in the basis) with z; — c; = 0, Thus 
the optimal solution is not unique. We know that there is one other basis 
that will yield the same optimal value of the objective function. Indeed, 
we can see immediately that there are only two basic optimal solutions. 
"This follows since, after a vector with ду — су = 0 has been inserted into 
the banin, 1; — cj = 2; — су, and the (2; — cj)-row in the new tableau 
with ay in the basis will be the same as in Table 4-10. There is no vector 
other than ay with г, — су = 0. Furthermore, the optimal solutions are 
not degenerate, and hence a vector with z; — су > 0 cannot be inserted. 

We can easily compute the new optimal basic solution. Note that 
Ar replaces а. Hence 


zm = 16.91 — БН (7.273) = 6.00 = ze, 


Bon dian = 40.0 = zy, 


fm = 6.308 + АШ (1273) = 10.0 = з, 


Of course, any convex combination of these two basic n 
will also be an optimal solution. ss optimal solutions 


5-8 Unrestricted variables. ‘Thus far our discussion of linear program- 
ming bas always been based on the assumption that the variables ту are 
ads t ba non-negative. On occasions, one encounters problems of 

linear programming type in which some or all of the variables сап have 
any sign. Variables which can be positive, negative, or zero are called 
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unrestricted variables. Wo wish to solve linear programming problems 
containing unrestricted variables. This can be done very easily by con- 
verting the problem to one having non-negative variables, Then the 
simplex method can be used without alteration. 

Suppose that in a given problem the variable zy is unrestricted in sign. 
Let us then write this variable as 


ar 4, xx AO (5-17) 


The variable т, has been written as the difference of two non-negative 
variables xj and х/. If xj > xy, then x; > 0, and if 2) < гу, it follows 
that т, < 0, Also, if zj = z/', then z; = 0. Hence, depending on the 
magnitudes of x} and x//, ху ean have any sign. 

By replacing a single unrestricted variable by two non-negative vari- 
ables, any linear programming problem involving unrestricted variables 
can be transformed into another linear programming problem involving 
only non-negative variables. If a, is the activity veetor corresponding to 
х; in the original problem, the new problem contains 

муб — xf) = ag + (7892, 
where the activity vector ay corresponds to т}, and the activity vector —а/ 
corresponds to z/. Similarly, the prices су and —e; correspond to 2 and 
j^, respectively. s 

We can now make an important observation, Any basic feasible solu- 
tion with non-negative variables cannot have т} and zy in the barie solu- 
tion, If both variables could be different from xero, then a; and —a; 
would be in the basis, However, ау, — а, are not linearly independent, and 
no basis can contain both of them. Thus in any optimal basio solution, 
either 27 = 0 and zy = xf, or zj« 0 and xj == zf, or x)= зу = 0 
and гу = 0. This demonstrates that the value of т, uniquely determines 
the values of xj, 27. If хт} > 0, then лу = О and vico versa. 


Examrie: Consider the following linear programming problem : 

хуа + та bory = b, maxs = Bry + 213 fii 

a; = (2,3), аз = [5,4], aa [1,0]; be (12,11). 
Assume that z; is unrestricted in sign. We write z, = z| — zy. The 


column vector corresponding to z} ія a, and that corresponding to ху ія 
—ay; the price of х; is З and that of ху is —3. In terms of non-negative 


variables, the new problem is 
ziant (а) + zaha + ла, = b, x4, 21,2225 > 0, 
maxs = 32, — 32 -+ 2rg + x. 
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This is a new linear programming problem in one additional varinble. If 
we write 24 = yi rj = ya 22 = Ys, T3 = Y4, then the linear program- 
ming problem to which we apply the simplex method is 


ЕТЕ 


тах 2 = 3y — 3ys + 2ys + уа. 


REFERENCES 
The references listed at the end of Chapter 3 apply also to thìs chapter. 


PROBLEMS 
5-1. Using the two-phase method, solve the following linear programming 
problem: 
За + 422+ Srst + zst xg = 8, 
a+ 3а 25+ Sra + 25+ zo = 3, 
2.521 + 5ra + 4.523 + 5.524 + 2.525 + dro = 7, 
all z; > 0, 
maxz = 21 -F 3r -+ 223 + 54 rs + Oz. 
5-2, Solve the following problem by the simplex method: 
da — za — rg fF r4 — 2x5 < 6, 
2e + 3224 dra — Sra d- zs € 9, 
zı та — 523 — 724 + 1125 < 10, 
mins = Злу + 22 — dza + Sra + Org. 


Assume that the variables za, zs are unrestricted in sign, while zi, z2, z4 are 
tive. 

5-3. We have shown in the text that if at some stage in thc simplex computa- 
tion, уу = 0 for all legitimate vectors ay and for an i corresponding to a column 
of B that contains an artificial vector, then row i of the tableau can be crossed 
off, and the artificial vector will never be removed. Show that this is true even 
if а legitimate vector is in column ї of В. Note, however, that in this case re- 
dundancy is not implied. 

5-4. Using the two-phase method or the — M. technique, show that if there is 
а tie between an artificial vector and a legitimate veetor, it is always desirable 
to remove the artificial vector. Show that this can be done without any fear of 
cycling. 
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5-5. Show how to solve a set of n simultaneous linear equations in n unknowns 
by means of the simplex method. Hint: Note that the variables are unrestricted 
in sign. Maximize z = —1x,, where x, is the vector containing artificial vari- 
ables. How does the simplex method indicate that the matrix of the coefficients 
is singular? What happens when there is no solution? 

5-6. Using the procedure developed in Problem 5-5, solve the following set 
of simultaneous linear equations by the simplex method: 


3zı 252 = 5, 
bry + 22 = 9. 
5-7. Solve the following set of simultaneous equations by means of the 


simplex method: 
521+ 322+ 23 = 2, 


411 = 22+ баз = 0, 
Өх + 722 + 423 = 1. 
5-8. Show how to invert a matrix by means of the simplex method. Hint: 


One can use any non-null vector for b. Why? What happens if the matrix is 


singular? 
5-9. Using the simplex method, invert the matrix 


Н! 


5-10. Using the simplex method, іпуег the matrix 


517 
А =|3 4 8| 
268 


5-11. Consider a linear programming problem of the form Ax = b, x > d > 0, 
maxz = cx, Instead of non-negativity restrictions, we now have lower bounds 
d; on each variable that can be positive. Show how to treat these lower bounds 
without explicitly including them as constraints, i.e., show how to convert the 
above problem to Fy = f, y 2 0, max gy, where F is m X n if A is m X n. 
Use this procedure to solve the following linear programming problem: 


221+ 822 — zs + dn. S 40, 
—2r + 223 + біз — 24 < 35, 
zı + 22 — 223 + 324 < 100, 
a 2 2,22 > 1,23 2 3,14 2 4, 
maxz = 32; -+ 12 -+ 223. 724. 
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5-12. Show how the simplex method can be used to solve a problem of the 


following type: 
Ах =Ъ, maxz = Delz]. 


‘The variables z; are unrestricted, and |x, is the absolute value of ту. Show that 
the same technique ean be used if the cost corresponding to a negative г; is differ- 
ent from that corresponding to a positive z; The activity vector for z; remains, 
of course, the same, regardless of whether x; is positive or negative. — ' 

5-13. Illustrate degeneracy in the extended requirements space. Show how 
it in possible to have z; — су < 0 with one or more yi; > 0, while z remains 

when a, is inserted into the basis. 

5-14. Suppose that we are given the following tableau for some linear program- 

ming problem: 


5-15. Find the second best and third best solutions to the problem descri 

RT y ^. we 
5-16, { the set of constraints Ax = b, b > 0, to a linear programming 
problem was obtained by adding a variable to each constraint, show 
that if by = maxby, and if A = Фу = оу — ау, i" k, daj = ayy, 
5 = (biss <, bal, Bi m be — bu, í y4 k, ba = ba, then Ax = 6, and only one 


of surplus variables is large, the simple transformation above can 
materially reduce the number of artificial variables, and this in turn might be 
expected to reduce the number of iterations. It is somewhat ising that 
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5-17. Interpret geometrically, in the requirements space, the criterion used 
in the simplex method to select the vector to leave the basis. In particular, 
give the interpretation for the example illustrated in Fig. 5-2. 

5-18. For the examples in Tables 4-2 through 4-4 plot, in the 
extended requirements space, the convex cones for each change of basis, Also 
plot the convex cone generated by all four vectors and show how finally reaches 
its maximum value, 

5-19. Solve Problem 4-19 by means of the two-phase method. 

5-20. Solve Problem 4-20 by means of the two-phase method. 


CHAPTER 6 
RESOLUTION OF THE DEGENERACY PROBLEM 


“The strawberry grows underneath the nettle, 
And wholesome berries thrive and ripen best 
Neighbor'd by fruit of baser quality.” 


Shakespeare—Henry V. 


6-1 Introduction. The general theory of the simplex method has shown 
that if any basic feasible, but not optimal solution to the set of constraints 
exists, and if degeneracy never appears, it is possible, by changing a single 
vector in the basis at a time, to reach an optimal solution (or an indication 
of an unbounded solution) in a finite number of steps. This follows because 
there is only a finite number of bases, and since at each step 2 > 2, no 
basis can be repeated. The step-by-step procedure terminates either when 
уу < 0, all т, for some a; with 2; — c; < 0, or when 2; — с; > 0 for 
all aj. The first case indicates the existence of an unbounded solution, 
while the second implies that an optimal solution has been found. 

_ . We have seen that degeneracy introduces complications into the above 
arguments. When degeneracy is present, the objective function may not 
change when we move from one basic feasible solution to another. Then 
we can no longer be sure that no basis will be repeated. In fact, we may get 
into a situation where we cycle forever, repeating the same sequence of 
bases, and never reach an optimal solution. Furthermore, there may be 
no way of avoiding this. In the following sections we shall show that it is 
possible to move from any initial basic feasible solution to an optimal 
basic solution without repeating any basis, i.e., that cycling need never 
occur, Indeed, a computational procedure that avoids the nossibility of 
cycling will be presented. - ^ 

We have noted previously that once we have shown that a basis never 
need be repeated, it immediately follows that one of the basic solutions 
will be optimal and that one of the optimal basic solutions will have all 
2) — cj 2 0. In this way, we prove by strictly algebraic means that one 
of the basic solutions is optimal even if degeneracy is present. Thus the 
theory is completely divorced from reliance on the geometric theory of 
convex sets. 

ie gps indicated, cycling never seems to occur in actual practice, 
although degeneracy is a quite frequent henomenon. 
praetieal point of view, nb not need M riis about Eur 
though 2 — z for a number of iterations, an' optimal solution will be 
reached in a finite number of steps, Examples where cycling does take 
174 
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place have been constructed, although in a very artificial way. Conse- 
quently, it is possible for a problem to cycle, and for this reason, it is 
desirable to develop a procedure that will ensure that cycling will never 
occur. Two different approaches to the resolution of the degeneracy prob- 
lem have been developed. One is the perturbation method of Charnes [1, 2]. 
The other, developed by Dantzig, Orden, and Wolfe [4], is a scheme involv- 
ing lexicographically ordered vectors. We shall discuss both methods. On 
the surface, these techniques appear to be quite different; however, in 
practice, it turns out that they are operationally equivalent. 


6-2 Charnes’ perturbation method. We shall first discuss Charnes’ 
perturbation method. All degeneracy problems arise from the nature of b. 
Degeneracy occurs because for some bases formed from the columns of A 
we do not need a positive amount of each basis vector b; in order to write b 
as a linear combination of the b;. Now if b were altered very slightly (this 
is usually referred to as “perturbing” b), then, when writing b as a linear 
combination of the basis vectors, we might hope that the perturbed value 
of b would require positive amounts of each basis vector for every feasible 
basis. This can be done, and it is exactly the technique which will be used 
to resolve the degeneracy problem. 

Instead of considering only the problem to be solved, we shall examine an 
associated set of problems whose requirements vectors, denoted by b(€), 
will differ slightly from the b of the original problem. When the perturba- 
tion is zero, this associated set of problems will include the original problem 
to be solved. We shall show that perturbed problems never become de- 
generate and that therefore no basis is ever repeated. The entire theory 
which we developed with the assumption that degeneracy does not appear 
can then be applied to the perturbed problem. Once the solutions to the 
set of perturbed problems have been found, it will be seen that the solution 
to the original problem is immediately available. We are therefore im- 
bedding the problem of interest into a whole set of related problems. 

Suppose that we replace the b > 0 of our problem by 


b() = b+ У ea; + 2; ее. (6-1) 


ji i=l 


We have added to b a polynomial in the number e, with vector coefficients 
involving the other vectors in A and any artificial vectors needed in the 
original problem. The number € will be taken to be positive and usually 
very small. A certain maximum allowable value of e, denoted by €max, is 
determined by the nature of the problem and is positive. It will never be 
necessary to compute Emax explicitly. However, the assumption will always 
be made that € < €max. We can then think of e as a variable whóse region 
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of variation isO < € < Emax. When € = 0, b(e) = b, and we are reduced 
to the actual problem of interest. In (6-1) €’ simply means e raised to 
the jth power. If, for example, A contained three vectors a;, a», a; and 
no artificial vectors were needed, b would be replaced by 


b(e) = b + ea; + eas + «аз. 


It makes no difference which vector from A is chosen as 8, the coefficient 
of e. Later, however, we shall find it quite desirable to have-the vectors 
which make up the initial identity matrix correspond to the lowest powers 
of e. When € > 0 and very small, then b(e) will differ only slightly from b. 
Any problem with b(e) for any e, 0 < є < €max, 88 its requirements vector 
will be referred to as a perturbed problem. 

Consider any basic feasible solution to the problem of interest. It can 
be written 


У tab; = b. (6-2) 
=1 
Because of the possibility of degeneracy, one or more of the cg; may be 
zero. Let us see what the solution will be if the basis is retained and b is 
replaced by b(e). If this solution is denoted by хв(є), it follows that 


Xn(€-) = Bo'b(e) = B^'b 4- » еВт'!а; + » e" "Bl; 
P =1 
ог 


nts 
09 = x» 4 S ey, 6-3) 
j=l 
If none of the xp; in (6-2) vanish, € can be chosen small enough (since a 
1 > gh (since zp; 
is independent of €) so that Xn(€-) > 0. On the other hand, if any xp; is 
zero, there may not be any € > 0 such that Xn(€) > 0. This depends on 
the signs of the yj. f 
For the moment, we shall assume that we have a nonde; i 
‹ 0 Л generate basic 
feasible solution to the perturbed problem, i.e., each of the basic variables 
is strictly positive. Later, we shall show how to find an initial basic feasible 
у. з) € Le ten aee with xp(e) > 0. It follows from (6-3) 
е е objective function z(e) for the soluti 
the perturbed problem is М c oou 


z(e) = свхь(є) = cpxp + ое 
j=l 


or 


(= e+ 93 ez. (64) 


i=l 
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Note that only b is perturbed; the prices remain unchanged. The e-terms 
appear only in xg(€), not in the yj. Hence, the z; — c; are the same for 
the perturbed problem and for the original problem. 

Given a nondegenerate basic feasible solution to the perturbed problem, 
хв(є) > 0,0 < є < Emax, we shall show that if this solution is not optimal, 
then it is possible to apply the simplex method without any risk of de- 
generacy, so that at each iteration 2(€) > z(e). After an optimal solution 
to the perturbed problem is obtained, we can set € to zero to find an optimal 
solution to the actual problem, At this point, the reader may be concerned 
at the thought of trying to apply the simplex method to the perturbed 
problem and having to worry about all the powers of є which appear. In 
practice, it will turn out that the cycling problem can be eliminated 
without ever using є explicitly. Only the tableaux that are constructed to 
solve the actual problem are needed. The є'з are merely used to prove 
that the final operational procedure actually prevents cycling. 


6-3 Selecting the vector to be removed. We are ready to change bases 
in the perturbed problem: The vector to enter the basis is computed in the 
usual way. If a, is to enter the basis and all ya, S 0, then there is an 
unbounded solution to the perturbed problem and to the actual problem 
(as we can see by setting є = 0). When опе or more у > 0, we must 
select the vector to leave the basis. As usual, this is done by finding 


zale) L min fo, ya > o) = «9. (6-5) 
Yrk i Yik 1 
If degeneracy is not to appear at the next iteration, this minimum must be 
unique. We are sure 0(є) # 0 because each zg(€) > 0. 

Let us show how the minimum in (6-5) is found. We can reformulate 
(6-5) to read 


2в.(6) _ |. Zhi NH i a» 0: (6-6) 
Vrk = in + бш 


It is important to note that when є is small enough (e < m3), only the 
lowest powers of € are important in (6-6). This implies that if min, (rni/ yin) 
is unique and is taken on at i = 7, then for € < Emax, the minimum in 
(6-6) is taken on at i = r. If the min; (zg;/ya) is not unique, the mini- 
mum in (6-6) will be taken on at one of the i-values for which zpi/ Vie is 
minimal; but in order to decide which i-value this is, the coefficients of 
the higher powers of є must be examined. Thus, if there is a tie for 
min; (z5;/y;z), then for those € for which z5;/ya is minimal, we examine 
the coefficients of e, that is, the Yi1/Yix- If for the i under consideration, 
min; y;1/yiz is unique and is taken on at i = r, then the minimum in 
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(6-6) is taken on for i = rif € < 6а. If again a unique minimum is 
not obtained, it is necessary to examine the coefficients of €, etc. We 
shall now prove in detail that this procedure for determining the minimum 
in (6-6) does yield a unique value and that the new basic solution will 
be feasible and nondegenerate, i.e., that for each #;(€) the first non- 
vanishing power of € has a positive coefficient. 

Consider 


o = min Е > o): (6-7) 
i (Yik 


If there is a unique minimum 2xg,/y,,, we remove the vector in column r, 
and the new basic solution will be feasible and nondegenerate. To see this 
note that the values of the variables for the new basic solution are 


nta | nts | i 
Bile) = їв: + Dy € yi; — (e +}, duy ) 35 
ici jc Urk 
nts 
= fsi t Dey, den (6-8) 


j=1 


L žer, SR ed NS d, ; 
#в(6 = NE 2 ёл = tart у) єў. (6-9) 

"There are several cases to be considered. First suppose that 00 > 0. 
Then for those 7 for which za; > 0 (this includes z5,), 25,(€) > 0 since 
£pi > 0. For those (if any) for which zg; = 0 and у < 0, 25;(€) > 0 
Since £p; > 0. For those û (if any) for which zg; = 0 and ya = 0, 
20:6) = тв(є) > 0. Since 00 > 0, there are no è for which zp; = 0 
and у > 0. Thus the new solution is feasible and nondegenerate. Now 
suppose that бу = 0, that is, zg, = 0. Let us first show that £p,(€) > 0. 
This follows immediately from (6-9). By assumption, хв„(є) > 0. Since 
ter = 0, it follows that the first y,; # 0 (ie., with the lowest index j) 
must be Positive. Then y;/yj is the first nonzero coefficient in (6-9) 
and it is positive since yp > 0, and hence for small nonzero €, £g,(c) > 0. 

When i > r, we see immediately that for those i for which xp; > 0, 
we have 25,(e€) > 0. For those i = r for which zgi = 0 (if any), it must 
be true that Jie < 0, for otherwise 60 would not be unique. If there exist 7 
with хві =0 and ya = 0, then £5,(e) = zsi(e) > 0. It remains to con- 
sider those i for which zg; = 0 and уь < 0. We must show that the first 
nonvanishing 9;; is positive. Recall that 

ыр ы ui) у: М 
i (Vii — Yri m 
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Because zg;(c€) > 0 by assumption, we know that the first nonvanishing 
yi; is positive. Also —yix/yrk > 0 since yir < 0. Denote by 7’ the lowest 
index j such that y,; > 0, and by j” the lowest index j such that y;; > 0. 
If j" <j’, then ў = 0, j< j; and ij” = yij" > 0 so that zra;(e) 
>0. If j' « j", jj; — 0, j € j', and ji = —уу'шь/у > 0 so that 
zpi(e > 0. Finally, if 7 = j”, fi; = 0, j < j”, and fj” > 0 since 
yi" > 0, yrj > 0. Thus 25;(e) > 0 in every case, and the new basic 
solution is feasible and nondegenerate. 

Next let us suppose that @ is not unique. If, in this case, the tie is 
resolved by an arbitrary choice of the vector to leave the basis, then 
ёрі = Oforall z r where the minimum is taken on in (6-7). We are not 
sure that for these i-values 2р;(є) > 0 since this depends on the sign of 
the fij, the coefficient of the lowest nonvanishing power of e. If we proceed 
by making an arbitrary selection, this ў;; can have either sign. Assume, 
however, that for the т tied in (6-7), we compute 


min e . (6-10) 


The smallest algebraic value is chosen (it may be negative because y;; may 
be negative). If there is a unique minimum y,1/y,z, we remove the vector 
in column r; then the new basic solution is feasible and nondegenerate. 
The new values of the variables are given by (6-8) and (6-9). The discus- 
sion of the case where 00 is unique shows that all variables are positive 
except the 7 # r tied in (6-7). We can show that if (6-10) yields a unique 
result, these xp;(€) > 0. First of all, for the i under consideration, £5; = 0 
because of the tie in (6-7). However, for these i # r, 


ба = уа — paa > 0, (6-11) 


since we have yi1/yik > Yri/yrk The coefficient of the first power of € 
in 2в{(є) for each such i s r is positive, and therefore, for sufficiently 
small e, 2в:(є) > 0. 

If there аге also ties in (6-10), the 0:1 are zero for the tied i 4 r; again 
we are not sure that the £5;(c) are positive. To resolve this problem, we 
move to the e?-terms and find 
min eu (6-12) 


i Wik 


for those 7 where a tie occurred in (6-10). If there is a unique minimum 
Yr2/Yrk, column r of the basis is removed, and the problem is solved. The 
above discussion shows that 2в:(є) > 0 for all i = r except those tied 
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in (6-10). Equation (6-11) with the subscript 1 replaced by 2 shows 
that the, variables xp;(€) [tied in (6-10)] will be positive if the minimum 
in (6-12) is unique. In this case, the variables z5;(e) will differ from zero 
only by terms involving є and higher powers of є. 

This process is continually repeated until we find some power of e for 
which there is no tie. It is certain that sooner or later we shall obtain a 
unique minimum. If we do not, i.e., if for every j a tie occurs for two or 
more values of î, then two rows in the tableau are proportional, for, if 
Yuj/Yuk = Yoj/Yor, all j, including the artificial vectors, then yu; = 
(Yur/Yor)Yo3, all j, and rows и, v are proportional. However, the original 
tableau contained an identity matrix, and all other tableaux contain 
B7" (see Problem 4-11). Hence there is a matrix of rank m in the tableau, 
and no two rows can be proportional (i.e., linearly dependent). We con- 
clude that, ultimately, the tie will be broken for some power of e. The 
vector to be removed is uniquely determined, and all the new basic variables 
are positive. Furthermore 


«9 = «(9 +2209 زم‎ — с) > 9. (6-13) 


The above analysis applies at every iteration. Hence degeneracy will 
never occur in the perturbed problem, and at each iteration 2(€) > z(e). 

Tf at each iteration 2(є) > г(є), no basis can be repeated. Since there is 
only a finite number of bases, the process must terminate in a finite number 
of steps. It can terminate only if (1) all y;; < 0 for some a; with 2; — 
cj < 0, or (2) 2; — с; > 0 for all legitimate vectors aj. If the terminal 
condition is (1), there is an unbounded solution to the perturbed problem 
and to the original problem. If the iterative process terminates with con- 
dition (2), then we know that an optimal solution to the perturbed problem 
has been found (Section 3-8). If we set є = 0, we also have an optimal 
solution to the original problem since varying є does not change the 
2j — 6j. 

Note that our discussion has proved that if there is an optimal solution 
to the original problem, at least one basic feasible solution will be optimal. 
Tn addition, there is at least one optimal basic feasible solution with z; — 
с; > 0 for all legitimate a;. This follows because if there is an optimal 
solution, the perturbed problem must end with all z; — c; > 0. Setting 
€ = 0 in an optimal solution to the perturbed problem yields an optimal 
solution to the original problem. We have also proved that it is possible to 
move from any basic nondegenerate feasible solution of the perturbed 
problem to an optimal solution in a finite number of steps. Using the same 
sequence of bases, we can also move from the corresponding basic feasible 
(but possibly degenerate) solution for the unperturbed problem to an 
optimal solution. 
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6-4 Order of vectors in b(e). We have yet to show how to find an 
initial basic feasible and nondegenerate solution to the perturbed problem. 
Let us number the vectors so that those forming the initial identity 
matrix come first, i.e., e, is the first vector, ez the second, ..., e, the 
mth. Thus the basic variables of the perturbed problem for the initial 
tableau are of the form 


zgi(€) = zaı +e + higher order terms, 
2в2(6) = «go + €? + higher order terms, 


.Bm(€) = zam + €" + higher order terms. 


This arrangement of the vectors shows that although some of the zg; are 
zero, each zg;(c) > 0 for small enough e. Hence, when we start out with 
an identity matrix for a basis, we can always obtain an initial basic feasible 
solution to the perturbed problem with хв(є) > 0. It follows that it is 
always possible to begin the solution to a linear programming problem 
with an identity matrix for the basis, and reach an optimal solution (or 
the indication of an unbounded solution) in a finite number of steps, that 
is, we can always prevent the occurrence of cycling. 

We have indicated that the value of є for the perturbed problem has to 
be in the range 0 < є < €max. We have not shown how to find €max. We 
compute €max by determining the largest possible value of € at each step 
which imposes the condition “for small enough €." Then eax will be the 
smallest of all these values. This procedure could require a great deal of 
work, but fortunately it does not need to be carried out. However, it is 
important to know that eax > 0, and that any ein the rangeO < € < €max 
will work for the perturbed problem. There is no limit on how small e is 
chosen to be, provided it is > 0; there is only a limit on how large є can be. 

Our discussion has shown that degeneracy causes difficulties only in 
determining the vector to be removed from the basis. If the proper vector 
is removed, cycling will never occur, regardless of how degenerate the 
basic feasible solutions become. However, we shall later present an example 
of cycling and see that if the wrong vector is removed, cycling can occur. 


6-5 Use of perturbation technique with simplex tableau format. To 
apply Charnes’ perturbation technique, only the tableau format ordinarily 
used to solve a linear programming problem is needed. It will be observed 
that e never really entered the computations at all. In a sense, the powers 
of € only serve to keep the vectors a; ordered. We always consider the 
lowest powers of e first since, for small enough e, they are the only ones 
that count. This means that we always consider the ratios y;;/yi with the 
smallest j first. To decide which vector is to leave the basis, we start with 


182 RESOLUTION OF THE DEGENERACY PROBLEM [снАР. 6 


the entries in the column headed "b" and compute min [z5;/yi.], yix > 0. 
If there is no unique minimum, we concentrate on the rows for which 
zpi/yik takes on the minimum value. For these rows, we move to the 
column headed “a,” and compute min {y;1/yi.}. If a unique minimum is 
not obtained, we turn our attention to those rows where y;1/yix has taken 
on its minimum value. For these rows, move to the column headed “ay” 
and compute min (yi2/yix], etc. Ultimately, there will be a unique mini- 
mum. If it occurs in row 7, this means that column r of the basis is removed. 
"The procedure is applied to each tableau as needed. The e's are never used. 
However, we must take care that the same column ordering is maintained 
from one tableau to the next. Of course, to use the perturbation technique 
within the tableau format, it is necessary to include in the tableaux, 
columns for any artificial vectors which may have been required. 


6-6 Geometrical interpretation. A geometrical interpretation of the 
perturbation method facilitates the intuitive understanding. "We have 
previously seen that several degenerate basic feasible solutions can cor- 
respond to the same extreme point in the solutions space. In the perturbed 
problem, there is no degeneracy, and hence only one basic feasible solution 
corresponds to each extreme point. This means that, in the perturbed 
problem, all degenerate solutions corresponding to a single extreme point 
are broken down so that each solution corresponds to a single extreme 
point. The degenerate extreme point of the original problem becomes a 
cluster of extreme points in the perturbed problem, with all the extreme 
points in the cluster lying very close to the point which was the original 
degenerate extreme point. The distance between these perturbed extreme 
points and the original extreme point depends on the size of e. As € ap- 
proaches 0, all the perturbed extreme points corresponding to a given 
degenerate point move closer and closer to this point until they all merge 
into the single degenerate extreme point. Note that the perturbation is 
such that the extreme points remain in the non-negative orthant. 

Note: Perturbation techniques similar to the types discussed here find 
applications in many other fields, For example, the removal of degeneracy 
in the energy levels of an atom by applying some perturbation such as an 
electric or magnetic field (Stark or Zeeman effects) is an illustration which 
arises in quantum mechanics, 

In the requirements space, any set of basis vectors generates an m- 
dimensional convex polyhedral cone. The cone generated by any k < m 
of these vectors lies on a face of the m-dimensional cone. When there is 
degeneracy, b can be expressed in terms of k < m of the basis vectors 
and thus b also lies on a face of the m-dimensional cone. The perturbation 
method moves b inside the cone generated by the basis vectors, and hence 
all the basis vectors are needed to express b(€) in terms of the basis vectors, 
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In fact, b(€) is inside every cone that yields a basic feasible solution to the 
problem. 


6-7 The generalized linear programming problem. The other technique 
for eliminating degeneracy is called the generalized simplex method. It 
uses lexicographically ordered vectors. We shall begin by defining a 
lexicographically positive vector. A vector x is lericographically positive 
(written x > 0) if its first nonzero component is positive, In other words, if 
we examine the components of the vector in dictionary order (from left 
to right), then the first nonzero component is positive. For example, 
(0, 2, —1, 4) is lexicographically positive since 2, the first nonzero com- 
ponent in dictionary order, is positive. On the other hand, (0, 0, —1, 7, 6, 3) 
is not lexicographically positive. Note that the symbol "x > 0” has a 
completely different meaning from “x > 0” which implies that each com- 
ponent of x is positive. By x; > xo we mean x; — X2,> 0. A vector is 
lexicographically non-negative (written x > 0) if it is either lexicographi- 
cally positive or if all its components vanish. 

To resolve the degeneracy problem, Dantzig, Orden and Wolfe [4] 
replace each variable z; of the original problem by a row vector x; with 
m -+ 1 components, the first component being wj. Then the requirements 
vector b is replaced by an m X (m + 1) matrix P given by 


P (bI. (6-14) 


The matrix P contains b in the first column, and the m, e; in the last m 
columns. Since P contains an identity matrix, r(P) = m. If the rows of 
P are denoted by p/, it follows from (6-14) that p! > 0 (i = 1,...,т), 
ie., the rows of P are lexicographically positive. The matrix P can be 
thought of as having for its columns 2, ..., (m + 1), the vectors from A 
or the artificial vectors which make up the initial identity matrix that 
would have been obtained if the problem had been solved in the usual way. 
The set of constraints Ax = b to the original problem now becomes 


aii + +++ алах = p^ = (bi, 1,0, +++, 0), 
EE ++ danin = р? (62, 0, 1,...,0), (6-15) 


Qm1X1 +. + GmnXn p" = (bm, 0,0,..., 1). 
If X is the matrix whose rows are the x;, then 
AX — P (6-16) 


is the set of constraints for the generalized problem. The matrix X is an 
n X (m+ 1) matrix. 
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Next, the objective function of the original problem is replaced by the 
vector function 


Zim) eT. (6-17) 


Umi 


Note that z is an (m + 1)-component row vector, and the first component of 
z is the z of the original problem. It will be observed that the prices have 
not been converted to vectors; they remain scalars. However, the price 
с; is now associated with the vector variable xj. 


6-8 The generalized simplex method. Using the ideas suggested above, 
we can propose the following generalized linear programming problem 
involving vector variables: Find a set of lexicographically non-negative 
variables x; 7 0 which maximize the z of (6-17) in a lexicographic sense 
and satisfy the constraints (6-16). By maximizing z in a lexicographic 
sense we mean that if z* is the optimal z, then z* > z forz corresponding 
to any other feasible solution of (6-16), i.e., any other solution of (6-16) 
with x; > 0 for all j. 

We shall now present the generalized simplex method, which is merely 
the simplex method as applied to the generalized linear programming 
problem outlined above. For the generalized problem, we shall show that 
degeneracy never occurs, and therefore all the related problems are elim- 
inated. Note that when an optimal solution to the generalized problem is 
found, an optimal solution to the original problem will be contained in 
the first components of the 2, ху. 

As usual, we shall denote by B a nonsingular matrix containing m 
columns from A. A basic feasible solution to (6-16) is one with no more 
than m of the x; lexicographically positive. The remaining n — m of the 
x; are null vectors. To be perfectly rigorous at this point, we should 
repeat the derivation of the basic theory of the simplex method for this 
generalized problem. We shall not do this since the arguments are identical 
with those developed for the ordinary problem. The details of some of 
these derivations are left for Problems 6-2 and 6-3. We shall proceed 
heuristically, i.e., the theory for the ordinary simplex method will be 
directly applied to the generalized problem. 

Any basic feasible solution to (6-16) can be written 
Xs = BP = (B^b B 'e,,... ‚В tem) = (xB, yu... Ym). (6-18) 
We have assumed that the vectors in the original problem have been 
numbered in such a way that the unit vectors €1,..., e, come first. 
Hence y; = B^e,, ete. We find an initial basic feasible solution to the 
vector problem by setting Хв = P. Recall that the rows of P are lexico- 
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graphically positive. It is assumed that any artificial variables which must 
be added to the original problem are also added as vector variables to the 
generalized problem. 

The 2; — c; corresponding to any a; in the generalized problem are the 
same as those for the basic feasible solution corresponding to B in the 
original problem. The vector to enter the basis at any one iteration is 
determined in the same way as in the ordinary simplex method and is, in 
fact, the same vector that would enter if we were dealing with the basic 
feasible solution xg to the original problem. 

Let us now consider how to determine the vectay to be removed. We 
use a straightforward generalization of the usual formula, that is, 


zi — min va > o}. (6-19) 
The difference appears in the computation of the minimum. Let us sup- 
pose that the vector variables xg; of the basic feasible solution (6-18) are 
lexicographically positive. We shall show that it is possible to obtain a 
minimum to (6-19) which is unique in the lexicographic sense. It then 
follows that all the new basie vector variables will be lexicographically 
positive. To determine the minimum in (6-19), we first compute min z5;/ 
Wi. Ifa unique minimum z5;/y;; is obtained, we remove column r from 
the basis. When a unique minimum is not found, we compute min y;1/yix 
for those z for which the zp;/y; assumed the minimum value. If there is 
still a tie, we move on and determine min y;2/yix for those т for which 
ут Гуль took on the minimum value. However, there cannot always be a 
tie since this would imply that two rows of Xg are proportional. However, 
r(B-^!) = m, r(P) = m whence r(X5) = m, and two rows cannot be 
proportional. Thus the vector to be removed will be uniquely determined. 
Next, we must show that the new basic vector variables are lexico- 
graphically positive. Clearly, a, = хв./у > 0. For the others, 


$2) RR E x pid Поет. (6-20) 
Urk 


Here again we must go through the type of analysis used in the discussion 
of Charnes’ perturbation method to show that each $5;(c) > 0, iz r. 
In fact, precisely the same analysis shows that each $5; > 0, i Æ r. We 
shall not repeat this material. 

The new value of the objective function 2 is given by 


2=2+ T (сь — гь). (6-21) 


But (хв,/унк) (сь — zx) 7 0, so that 2 > z. In the lexicographic sense, 
there has been an increase in the objective function. At each iteration, 
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the objective function increases lexicographically. Hence we conclude 
that no basis can ever be repeated. The xp; = (xpi, yi; ... ;Vim) are 
uniquely determined by the basis. If we have two bases with the same 
veetors from A, and these vectors are in different column positions in B, 
then the rows in B7" and the rows of Xp are changed around, but the 
xg; are not affected. Consequently, the z for any basic feasible solution is 
unique. 

Thus we conclude that the process can terminate only if (1) y;; € 0, 
all 7, for some a; with 2; — с; < 0, or (2) zj — cj > 0 for all legitimate 
vectors aj; in the latter case, the basic feasible solution is optimal. Since 
any basic feasible solution to the generalized problem contains within it 
(in the first components of xg; 2) a basic feasible solution to the original 
problem, we have proved that it is possible to move from any basic feasible 
solution of the original problem to an optimal basic feasible solution by 
changing a single vector at a time. We have also proved that there is at 
least one optimal basic feasible solution to the original problem, with all 
2; = с; 2 0. 

The reader may be disturbed at the suggestion of carrying out a com- 
putation involving (m + 1)-component vectors by means of the generalized 
simplex method. A little thought, however, shows that the impressive 
notation was necessary only to prove that a certain way of selecting a 
vector to be removed really does prevent cycling. We do not need to use 
the generalized simplex method to prevent cycling when degeneracy occurs. 
We only have to arrange the columns of the usual simplex tableau so that 
the eolumns associated with the original identity matrix come first and in 
the proper order. Then, if at any stage in the ordinary simplex method 
min zp;/yiy is not unique, we move to the next column and compute 
min yi1/yix for those ? for which z5;/y;y assumed the minimum value, ete. 
In this way, the vector to be removed will be uniquely determined, and 
cycling will never occur. An optimal solution will be reached in a finite 
number of steps. 

We have already mentioned that although Charnes’ perturbation method 
and the generalized simplex method appear to be quite different, they are 
operationally equivalent. That is, the rule for determining the vector to 
be removed in the ordinary simplex tableau is identical for both methods, 
provided the vectors forming the original identity matrix come first in the 
tableau. Recall that the e"s in Charnes’ method only served to introduce 
а certain order. Dantzig, Orden, and Wolfe took advantage of this ob- 
servation and developed an arrangement using lexicographically ordered 
vectors rather than powers of €. At this Point, we also wish to note that 
Charnes, in replacing b by b(c), did not really have to include all the 
vectors in the tableau. He needed only those vectors that make up the 
original identity matrix. 
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6-9 Examples pertaining to degeneracy. The preceding discussions may 
have led the reader to believe that at any stage in the solution of a linear 
programming problem, there is always a vector which, if the optimality 
criterion is not satisfied, can be inserted such that 2 > z. This is not true. 
There may not be any vector to be inserted that will actually increase z. 
To be able to jump to an adjacent extreme point which increases z, it 
may be necessary to change several vectors in the basis. 


ExaMwPLE: Consider the problem given in Table 6-1. No vector can be 
inserted such that 2 > z. Only aj, аз are not in the basis. Both can be 
inserted, but each enters at a zero level. It will be noted that 


ба + даа + 2004 + a5 = b, any 0 > 0, 


and therefore z can be made arbitrarily large. Thus it is possible to in- 
crease z, but at the first iteration z cannot be improved. 


TABLE 6-1 — INITIAL TABLEAU 


e; 2 3| 0 0 0 

Vectors 
Cp in b ја | a2 | аз | 84 | 85 

Basis 

0 a3 0 2|—4| 1 0 0 
0 a4 0 |-38 210 1 0 
0 a5 1 0 0| 0 0 1 
2;—c6|0]|—2|—3]| 0 0 0 


ІБ NS 


It might also be thought that if the optimality criterion is not satisfied, 
then there must be an increase in z before it will be satisfied. This is not 
true. It is possible to insert a vector at a zero level, with z remaining 
unchanged, and move from a solution where the optimality criterion is 
not satisfied to one where it is. 


ExaMPLE: Consider 
2,81 ха: + хзаз = b, allz; 2 0, тах 2 = сүү + Cote + C323, 
ау = [1,0), аг = [0,1, аз = [2,1], b= [0,1], 
Cy A сз 258, €3 — 10. 
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If the initial basis is В = (ау, аз) = I, then 23 — єз = —3, and at the 
next iteration ag enters the basis, and ау is removed. The original basic 
solution is хв = [0, 1], and the new basic solution is £5 = [0, 1] since 
аз enters at a zero level. Hence 2 — z. The new basis matrix is B = 


(аз, 42), and 
sal 1 |J: 
-t 2 
thus 
тек 
=1 2}lo т 
and 


4—623010—3)—2-— 15. 


Hence for the new basic solution the optimality criterion is satisfied, 
although this was not the case for the original solution. However, the 
solution x and z did not change. 


It will be helpful to give an example illustrating the procedure for 
uniquely determining the vector to be removed, which assures us that 
cycling will not occur. At the same time, we might note that the simplex 
tableaux constructed to solve the original problem essentially contain the 
solutions to Charnes' perturbed problem and the solution of the generalized 
simplex method. Recall that for Charnes' perturbation method, 


nts nts | 
tail) = zpi + Уу еу; 26) = 2+ У es. 
je ji 
The tableau provides us with the xpi, уу, z and, from z; = (2; — cj) + cj, 
the zj. Thus the tableau yields all coefficients of the polynomials appearing 


in the perturbation method, However, we are not supplied with the value 
Of Emax. 


For the generalized simplex method, the vector variables and the vector 
objective function are given by 


Xai = (rni Var, +++, Yim), 
Z= ©вХв = Cp(XB, yy... , yo) = (e zy. 3 2m): 


Thus, at any stage, the usual tableau supplies the complete corresponding 
basic feasible solution to the generalized simplex method. 


EXAMPLE: ‘Consider the linear programming problem whose initial 
tableau is Table 6-2. On the first iteration, a4 will enter the basis. 
When min z;/y;4 is computed, a value 0 is obtained for 252/24, tB3/J34- 
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TABLE 6-2 — TABLEAU 1 


The vector to be removed is not uniquely determined from this computa- 
tion. We then move to the a;-column and determine y21/J24, Va1/V34- 
Both of these are 0 also. Next we move to the as-column and find y22/ 
424 = $, уз2/0за = 0. Here a unique minimum is obtained, and column 
3 of the basis is removed. "The new tableau is given in Table 6-3. 

From the information given in the tableaux, let us write down the 
corresponding solutions for Charnes' perturbation method and the gen- 


TABLE 6-3 — TABLEAU 2 
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eralized simplex method. For the first tableau, we find that for Charnes’ 
scheme 


tale) = 2+ e+ 26 + 4e, xpo(e) = 6 + 3e + е5, 
твз(є) = € + 4e* + 265, 


and since 2; = г; — cj + cj, z(e) = 0. The new values obtained on 
inserting a4 are simply read off the second tableau. They are 


$n(0—2-e—idé-F3é, fpo(e) = e — $8 — 46), 
2вз(6) = de + ef + $e, 
For small enough e, the тв;(є) are indeed positive. Note that zp»(6), 
2в3(є) differ from zero only by terms involving €, j > 1. Also, #(є) = e3 + 
4e* + 2e, and 2(6) > z(e). 
The generalized simplex method yields for the initial solution 


X» = (2,1,0,0);  xn2 = (0,0,1,0); хрз = (0,0,0, 1); 
z = (0, 0, 0, 0). 


The basic vector variables are lexicographically positive. For the second 
tableau, 

251 = (2,1,0,—4), Яз = (0,0, 1, —3), 

255 = (0,0,0,4), 2 = (0,0,0, 1). 


The new basic vector variables are also lexicographically positive and 
202—1, 


6-10 An example of cycling. Examples have been constructed for which 
cycling will occur if the wrong vectors are removed from the basis. These 
examples are quite artificial and were constructed solely for the purpose of 
demonstrating that cycling can take place. Specifically, there exist at 
least two examples: One was developed by Hoffman [5] and the other by 
Beale [1]. The example used here is one suggested by Beale.* The reader 
should keep in mind that the example was constructed solely to demon- 
strate cycling, and that no actual problem has ever cycled. 

Consider the linear programming problem 


0.2521 — 8123 — z4 + 9x4 < 0, 
0.54; — 12r, — 0.523 + 324 < 0, 
Ta < l, z1,25,25, 4 > 0, 
maxz = 0.752, — 20r, + 0.523 — 6z4. 


(6-22) 


* In actuality, this example is the dual of one suggested by Beale. 


5 
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The first tableau is given in Table 6-4. We have arranged the vectors so 
that those forming the initial identity matrix come first. This will allow 
us to use the procedure which we have developed for the purpose of uniquely 
determining the vector to be removed. At the first iteration a; is inserted, 
and either as or ag can be removed. If we move to the a;-column and 
compute yi5/yi1 = 4, 25/021 = 0, we find that ag should be removed if 
we wish to ensure that cycling will not occur. 

Suppose that instead of ag, as is removed. By properly choosing the 
vectors to be removed from the tied values, we can obtain the sequence 
of tableaux given in Tables 6-4 through 6-10. Note that tableau 7 is 
precisely the same as tableau 1. Hence we have cycled and come back to 
our starting point. If we continue to remove the same vectors, we shall 
cycle forever, and an optimal solution will never be reached. 

Beginning with Table 6-10, which is the same as Table 6-4, let us remove 
ag instead of a5; ав is the only other vector which can be removed. We 
then obtain Tables 6-11 and 6-12. Observe that in tableau 8, there is no 
tie for the vector to be removed. An optimal basic feasible solution is 
obtained in tableau 9. 

It is interesting to note that if we had not resolved the ties by the 
method which ensures that cycling will not occur, but had instead used the 
simple rule prescribing that we remove the vector for which yix is the 
largest among the corresponding to the tied vectors, we would have 
removed as, and cycling would not have occurred. 


TABLE 6-4 — TABLEAU 1 


ej 0 0 0 075 | —20 05 | —6 
Vectors 
св іп b | as | a6 | a7 al a2 as а 
Basis 
0 a5 0 1 0 0 0.25 —8 | -1 9 
0 ав 0 0 1 0 0.50 | —12 | —0.50 | 3 
0 a7 1 0 0 1 0 0 1 0 
ر2‎ — ĉj 0 0 0 0 | —0.75 20 | —0.5 6 
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PROBLEMS 


6-1. Solve the following linear programming problem by the simplex method. 
Determine the vector to leave the basis in such a way that degeneracy will not 
occur. Does this method yield an answer different from that obtained by select- 
ing the vector for which ууу is the greatest of the tied values? If there is a differ- 
ence, compute the sequence of tableaux for both methods. 


1.521 + 0.512 3x3 + 24+ 425+ 2:6 > 0, 
2х1 + 3224 0.5x3 + 424+ 25 776 > 0, 


ау Are— тз — 814 625 — ze < 0, 
0:521 — 212 — 1.513 + 4z4 + 315 — бл < 1, 
allz; > 0, 


maxz = 521 + 622 + 3z3 + z4 — 2x5 + 76. 


6-2. For the generalized simplex method involving vector variables, prove 
that if there is a feasible solution, there is a basic feasible solution. 

6-3. For the generalized simplex method, prove that if for some a; not in 
the basis 2; — cj < 0, then if all yi; < 0, there is an unbounded solution 
(what is the meaning of an unbounded solution ?), and if one or more yi; > 0, 
there is another basic feasible solution with 2 > z. Thus show that if there is 
an optimal solution, one can move by changing a single vector in the basis at a 
time from any basic feasible solution to an optimal basic feasible solution in a 
finite number of steps. Also show that any optimal basic feasible solution has 
all z; — c; > 0. 

6-4. Show that cycling can never occur so long as a unique minimum is 
obtained in the computation of min; tpi/y > 0. Show that this is true even 
if tpr/Yrr = 0 and degeneracy occurs, 

6-5. After having studied this chapter, 
Problems 4-3 and 4-4? 

6-6. Give a general proof for Problem 4- 
degeneracy assumption. 


can you provide a better answer to 


10 which does not require the non- 


CHAPTER 7 
THE REVISED SIMPLEX METHOD 


“Though this be madness, yet there is method in’t.” 
Shakespeare—Hamlet. 


7-1 Introduction. The revised simplex method was developed by 
Dantzig, Orchard-Hays and others at the RAND Corporation as an 
efficient computational procedure for solving linear programming problems 
on digital computers. The revised simplex method solves a linear program- 
ming problem in the same way as the simplex method. Given any basic 
feasible solution, we move from that solution to'an optimal basic feasible 
solution by changing a single vector in the basis at a time. For the revised 
simplex method, the criteria used to determine the vector to enter and 
leave the basis are precisely the same as those used in the simplex method. 
The “revised” aspect concerns the procedure of changing tableaux. In dis- 
cussing the tableau format for the simplex method in Chapter 4, we noted 
that it was unnecessary to transform all the уу, Xs, 2; — cj, г at each 
iteration. In fact, all new quantities can be computed directly from their 
definitions if B^! is known, i.e., if only the basis inverse is transformed. 
The revised simplex method makes use of this observation. At each 
iteration, only B7", xp, свВТ!, and z are transformed (z, xp, cB" are 
also transformed since this requires less computational effort than their 
computation from definitions). None of the y; is transformed. In fact, the 
only у; that is ever-determined at each iteration is the one for the vector 
which is going to enter the basis. As we shall see later, this type of trans- 
formation scheme has several advantages over that used in the simplex 
method. 

There are two standard forms for the revised simplex method, depending 
on whether it is necessary to add artificial vectors to obtain an initial 
identity matrix. For Standard Form I, it is assumed that an identity 
matrix is present after slack and surplus variables have been added, and 
hence artificial variables are not needed. If artificial vectors must be 
added, then Standard Form II is used. As might be expected, the revised 
simplex method uses the two-phase technique to handle artificial variables. 
However, it does so in a way slightly different from that discussed in 


Chapter 5. 


7-2 Revised simplex method: Standard Form I. In the revised simplex 
method, the objective function is essentially treated as if it were another 
constraint. Where we dealt with an m-dimensional basis in the simplex 
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method, we deal with an (m + 1)-dimensional basis when using Standard 
Form I of the revised simplex method. There is a good reason for doing 
this, which we shall point out later. 

Our usual formulation of the linear programming problem would be 
Ax = b,x > 0, maxz = cx. Instead of writing z = cx, let us write 

2— CX = 2 — Citi — ۰۰۰ — Cat, = 0. (7-1) 

This can be considered as another constraint equation for which z is to be 
made as large as possible. If we now write 


2 бүйүз си. -— 0, 
Oui + +++ + aiv. = by, (7-2) 
Omit) F +++ + mnis = bm, 


then (7-2) can be considered to be a system of m + 1 simultaneous linear 
equations in n + 1 variables z, Z1, ... , т. We wish to find a solution to 
(7-2) such that z is as large as possible (and unrestricted in sign), subject 
to the non-negativity restrictions z; > 0 (j = i 
Equation (7-2) can be rewritten in a more symmetrie notation, as 
follows: 
To + doit, +++++ aont, = 0, 


Фу +++ алата = by, (7-3) 


amita + +++ аһа = bm, 


where 2 = zo, —c; = ao. In partitioned matrix form, this can be written 


DERE 


where a^ = (ao1,...,ao,) and A is the A of Ax = b. In terms of our 
original notation, (7-4) would be written 


$ i ` |] C N ; (7-5) 


Equation (7-2) or (7-3) is referred to as Standard Form I for the 
revised simplex method. A linear programming problem with an identity 


matrix in A is cast into this form when it is to be solved by the revised 
simplex method. 
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Corresponding to each activity vector aj of A, we can define a new 
(m + 1)-component vector* 


aP = {-c;, aj] = [aop aj, 7=1,...›% (7-6) 
and corresponding to b, we can define the (m + 1)-component vector 
Ъ‹ = [0, b]. (7-7) 


The column corresponding to 2 = to is the (m 4- 1)-component unit 
vector €1. 

A basis matrix for the set of equations (7-4) or (7-5) will be of order 
m + 1. We are seeking a basic solution to (7-4) or (7-5), with one 
of the basic variables being z = то and the other m basic variables xp; > 0, 
such that, subject to the non-negativity restrictions, zo is as large as 
possible. Since we are trying to maximize zo, this variable must be in 
every basic solution, i.e., the vector e; corresponding to zo must be in 
every basis matrix. We shall assume that it is always in the first column. 
An (m + 1) X (m + 1) basis matrix containing e; and m linearly inde- 
pendent vectors af? will be denoted by В}, where 


B, = (ey, bi. .., b) = (0, bi... bi). (7-8) 


As in the simplex method, the b(? refer to the columns of Bi, and we 
must know by other means which a? is in column 7 of Ву. Clearly, e; and 
any m vectors af? = [ао aj] such that the corresponding aj are linearly 
independent will form a basis for "+! since there is no linear combination 
of the a‘ which will yield a vector whose last m components are zero 


unless the scalar coefficient of each aj? is zero. 


There is a one-to-one correspondence between the basis matrices Bı 


for the set of constraints (7-4) and the basis matrices B for the set of соп-. 
straints Ax — b. In fact, it is very useful to write any B, in the partitioned 


form 
Bie i ep (7-9) 


* Some notational difficulties are encountered in the: discussion of the revised 


simplex method. For Standard Form I, all vectors have т + 1 components, 


rather than nents, as was customary in the simplex method. Similarly 
Wc +1. For Standard Form П, 


the basis matrix in Standard Form I is of order m = ‘or § 
all vectors have m- 2 components, ‘and the basis matrix is of order m + 2. 


A superscript (© on vectors will indicate that they have m piel, components, 
and that they are the vectors for Standard Form I of the revised simplex method. 
A subscript 1 on a basis matrix will indicate that it is of order m+ 1. A super- 
script (® on vectors and а subscript 2 on the basis matrix will indicate the quan- 
tities appropriate to Standard Form II of the revised simplex method. 
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where B is the basis matrix for Ax — b containing those columns a; from 
A which are in the columns af? that appear in Ву. Equation (7-9) shows 
how to move from any basis matrix for Ax = b to the corresponding basis 
matrix for (7-4) or (7-5), and vice versa. 

'The results of Chapter 2 on the partitioned form of the inverse allow 
us to write down immediately the inverse of (7-9). We see that 


Вг! = [ М (7-10) 
om? 
We can write any af? as 

aj? = уо) + ybi? + +++ 4 уа БО, (7-11) 

or if 
Y)? = [oss <-s уюл, (7-12) 

then 
yj? = Bre. (7-13) 


However, suppose that we use (7—10) for Вг! and (7-6) for а), This yields 


ур [, aii Ий dr gen е |а|. qao 
ОВ aj Bola; yi 


We have obtained the interesting result that the first component of 
yf" is z; — cj, and the last m components make up yj. Here we have the 
reason for treating the objective function as one of the constraints, To 
find 2; — c; for any a;, we simply form the scalar product of the first row 
of Вг! with aj”. This requires only m multiplications. We shall see that, 
in moving from one iteration to the next, we have to perform only m + 1 
multiplications to obtain the first row of By". Thus, in order to compute 
п — т of the г; — c; only nm — m? + m + 1 multiplications are re- 
quired. However, if we transformed only B~!, we would have to compute 
cyB' at each stage, and this would require m? multiplications, so that 
nm multiplications would be necessary to determine n — m of fhe ey — e. 
Unless m is negligible with respect to n, we avoid a considerable amount 
of computation by using the larger basis, In reality, all that is accom- 
plished by using the larger basis js that ©вВ-! as well as B are trans- 
formed at each stage [see (7-10)]. 

Ё Мое: When discussing the number of arithmetic operations involved 
in any computation, we shall usually count only multiplications and 
divisions, because, by hand and on a digital computer, multiplication and 
division take considerably more time than addition or subtraction. 
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If we define the (m + 1)-component vector xf by 
xj = Bb, (7-15) 


then х{2 is a basic solution for (7—4) corresponding to the basis matrix By. 
From (7-7) and (7-10), we see that 


Р ro 
OBA ОБ Bob Xp 


As expected, xy contains as its first component 2 and as the remaining m 
components the тв; of the basic solution for Ax — b corresponding to 
the basis matrix B. 


7-3 Computational procedure for Standard Form I. It should now be 
fairly clear how the above definitions are used in solving a linear program- — 
ming problem. To apply Standard Form I we must assume that an 
identity matrix appears in A. For the initial basis matrix for the system 
of constraints (7—4), we use the columns aj? for which the a; yield the 
initial identity matrix in A. The inverse of the first basis can then be 


written 
; B=]! ad (7-17) 
0 In 


and if the columns from A making up Im correspond to slack variables, 
св = 0. The initial basic solution to (7-4) is xf) = (cab, b]; it is feasible, 
since a feasible solution to (7-4) is one for which the last m components of 
xf are non-negative; the first component 2 can have any sign. „Ме now 
have a basic feasible solution to (7-4) and the inverse of the basis matrix. 
To initiate the revised simplex method, we compute the z; — c; for each 
P not i i i lar product of the first row of BT 
a” not in the basis by forming the scalar produc i 


с H 
with each а, "The vector aj? to enter the basis is determined from 


Ey — ey min Q;j—e 2—9 < 0. (7-18) 


smallest index j. Next we must deter- 


If there is a tie, we can choose the 1 
mine the vector to be removed. Recall that we always wish to have 
lumn of В; is never a candidate 


то = zinthe basis. Therefore, the first со 
for removal. Let us number the columns of B; by 0, 1, ..., m so Ws the 
first column has the index 0. At the present time we do not have уй. We 


now compute yf? from yz) = Brat = [ж — с Yal- 
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The vector to be removed is then determined in the usual way. We 

compute 

Br . [TBI 

= min Pe ya > 0), (7-19) 
and remove column r of B, (if the physical columns were counted, this 
would be column r + 1, but by our numbering it is column т). A tie can 
be broken in any of the ways discussed previously. 

At this point, it might be noted that Вг! provides all the information 
necessary to ensure that cycling will not occur. The identity matrix from 
A appeared in the first B, in columns 1, . . . ‚ т. Therefore columns 1, . .., 
m of B; ` are equivalent to the first m columns in the usual simplex tableau, 
provided the columns making up the initial identity matrix comé first in 
the tableau. Consequently, columns 1, ..., m of Вг! can be used to 
determine uniquely the vector to leave the basis, 

Having obtained the vector to enter the basis and the vector to leave the 
basis, we are now ready to perform the transformation to the new basic 
feasible solution of (7-4), where the first component of xf? is at least as 
large as the preceding one. In the revised simplex method, only the 
quantities xj? and By" are transformed. We transform x as well as 
BT’, since only m + 1 multiplications are required to transform xj, 
while its computation from the definition x) = B; !b, requires m(m + 1) 
multiplications. 

If the new inverse is denoted by Вг! and the new basic solution by #0, 
then, using the theory developed in Chapter 2, we see that 


TSB Br, 
7-20) 
$5 = E;Bi'b? = Exp, 
Matrix E, has the form 
Ерке (OD nen 1 Or 26m 13); (7-21) 


We must, be careful to notice that n” appears in column r + 1 of Ej. 
This is the case because the actual physical column of B, which is being 


removed is r + 1. We called it column r of Bi, since we numbered the 
columns 0, 1,...,m. Then 


(D IT eee OL Yik 1 
E ce ET параны онад, x 
V Yre” "yas i 4 0520 


It is a little more convenient to apply a transformation formula of vector 
type (like that developed for the Simplex method) than use multiplica- 
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tion by Ey. Suppose that we denote the last m columns of Bi! by 8i?;... 
(? and 8j? = [8oj, B15 - - - ; nj]. Then 


ÊT! = (ei ÊP, ..., Ê) = E;Br! = (ex, E:8\”,..-E:Bm’), (7-23) 
and hence 
AS? = Eg, j= 1,:...ту $9 = Ех. (7-24) 


Let us write Ф = n” — e,,,; then 


AY = gi + gue, P = xP + x20, (7-25) 
where 
о а Чаке sb FORE MS „йы. | 
Ф [ Vx › "n , jn ds; FE (7-26) 


The “ring around the rosy” method of transformation can also be used 
here for the following reason: If we write 


Boo = z, Bio = zp = 1,...,т; уок = 26 — Ck, (7-27) 
then the transformation formulas (7-25) can be written 


Ви = Ву LE Beg $0.5. m ix*r j-0..,m 
Urk 
(7-28) 


&,- B. (7-29) 


We have shown how to transform Вг! and xf’ when we move from one 
basic feasible solution to the next. Using the new By’, we find the new 
2р — cj by forming the scalar product of the first row of Вг! with the 
af? not in the basis. If all 2; — c; are not > 0, the whole process is 
ове 

The computations for the revised simplex method can also be arranged 
into а convenient tableau form (see Table 7-1). Note that the first column 
of Вг! is always eı. Therefore, we never need this column in the tableau 
and we never need to transform it. We only need columns 8f", . . . , 8%) 
of B; !, x, and yf". It is somewhat more complicated to use this tableau 
form than that for the simplex method because the given tableau does not 
contain all necessary information. In addition to constructing a tableau 
fs Table 7-1 at each step, we must have a tableau containing the vectors 
a 

In moving from one iteration to the next, we first fill in the columns 
BP, ..., В0), x9 of the new tableau, using the transformation formulas 
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(7-25) or (7-28) and (7-29), together with the pertinent information in 
the preceding tableau. The y{’-column is not filled in at this point. We 
next use the first row of the new tableau and the vectors af? in the addi- 
tional tableau and perform some side computations on “scratch paper” 
to find the z; — cj. This determines the vector to enter the basis at the 
next step. We can now fill in the y{?-column, since af? has been determined. 
To find у, we use yj? = Вг а); this computation is also made on 
scratch paper. Then, the vector to be removed can be obtained from the 
numbers in the last two columns (if one wishes to use the rigorous method 
to ensure that cycling will not occur, the first m columns in the tableau 
may also be needed). 

The above procedure is the revised simplex method for Standard Form I. 
This form can be used when no artificial vectors are needed. It should be 
noted that if a problem is solved by the revised simplex method, it is not 
easy to find the corresponding tableaux for the standard simplex method. 
The revised simplex method does not compute all the y;, which, if needed, 
must be computed from B; ‘aS’. It is easy to move from a tableau for the 
standard simplex method to one for the revised simplex method because 
ВТ! is available, and we need to compute only the first row c5B- of the 
revised simplex tableau. 


TABLE 7-1 


TABLEAU Form or THE REVISED SIMPLEX METHOD 


Sranparp Form I 


Variables 
in 
Basis 


zo 
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7-4 Revised simplex method: Standard Form II. Standard Form П is 
used when artificial vectors must be added to obtain an identity matrix for 
the initial basis matrix. The two-phase method is used to handle the 
artificial variables: In Phase I, the artificial variables are driven to zero, 
and in Phase II, an optimal solution to the original problem is found. 

We have noted previously that if some artificial variables appear in the 
basis at a zero level upon termination of Phase I, special precautions 
must be taken to make sure that these variables do not become positive in 
Phase II. In Chapter 5 we have outlined one procedure for keeping the 
artificial variables at a zero level in Phase II. 

Dantzig and Orchard-Hays do not use that procedure in the revised 
simplex method. Instead, at the beginning of Phase II, they annex the 
additional constraint equation 


УУ та; = 0. (7-30) 


iml 


Since the simplex method always maintains each za; > 0, Eq. (7-30) 
requires that all artificial variables be zero. 

In the discussion of Standard Form I, we have seen that it is very con- 
venient to treat the function to be optimized as one of the constraints. Let 
us use this approach in setting up the two-phase method. To be specific, 
it will be assumed that an artificial variable was added to each constraint 
equation. 

In Phase I, we have the following set of equations: 


2* + ga + +++ + Zam =0, 
Tar х Ж RO Vota = bi, (7-31) 


Zam + mii + +++ + mnn = bm. 


We wish to find a basic feasible solution to (7-31) with z* in the basis, 
which maximizes z* (recall that a basic feasible solution to (7-31) does, 
not require that z* be non-negative). If we write 2* = 54.1, Та = n42, 
..., Фат = nm, then (7-31) can be written in the more symmetrical 
form 
Enyi F Tna +++ + nm = 0, 
aun Tot ап uc = bi, (7-32) 


Gmiti T^i + амал» + тафта = bm, 


and z,44 is to be maximized. 
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If max z,4, = 0, we go on to Phase II. When MAX an +ı < 0, there is 
no feasible solution to the problem. Let us assume that max Cnet 0; 
and that one or more artificial vectors may be in the basis at а zero level 
when Phase I terminates. Then in Phase II, we deal with the following 
set of equations: t 


2 — сл — +++ саль = 0, 
Tape hel dri 0, 
anzi cod aint, F Tapa =b;, (7-33) 


Umi, FH ++ + amnia Filati i= bm. 


"The second constraint equation is the one added to ensure that the artificial 
vectors remain zero. It is convenient to increase the row index by one 
and to rewrite the original constraint matrix А and the requirements vector 
b as follows: 


|" а» 
А ~ |: t ' = 3100; | 
Om 1,15 ** Om tn yir is Ji 


"This permits us to write (7-33) in the more symmetrical form 


Tot aozi — + +++ aar, = 0, 
Tapa К+ Хату = 0, 
бйз NEO o4 Tr42 = by (7-34) 
Ome titi bees + Om pi nln T nma = 0,4. 


We can now note that, aside from the variable z, i 
that, +1, (7-34) is merely 
(7-32) plus an additional equation. However, at hs end of Phase I, 
7541 = 0; hence, we could include it in (7-34) without changing anything, 
since the simplex method will always maintain it at a zero level. This 


Zo + aozi + +++ + ал, = 0, 
аы Ка eee Tr+mt1 = 0, (7-35) 
anti + +++ 4- аыр, + tn42 = be, 


анын t ib Gast ate + лыда = бз. 
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At this point, the following question may be asked: "Why not use the 
single set of equations (7-35) in both Phase I and Phase II?" In Phase I, 
we maximize 2,4; (and ignore zo), and in Phase II, we maximize ro. 
During Phase I, all variables except zo, Zn 4.1 are required to be non-negative. 
During Phase II, z is the only variable which does not need to be non- 
negative. This means that, in Phase I, the first equation involving the 
actual function to be optimized is merely carried along, and if no artificial 
vectors appear in the basis at the end of Phase I, then in Phase II, the 
same is true for the constraint which maintains the artificial vectors at 
a zero level. This is precisely what is done. The set of equations (7-35) is 
referred to as Standard Form II for the revised simplex method. 


7-5 Computational procedure for Standard Form П. We are now ready 
to examine the details of the computation by means of Standard Form II, 
Let us denote the column vector corresponding to any variable z; in (7-35) 
by 


aj? == [aos arj» y ант (7-36) 
For legitimate vectors, 
aS = (—0;,0,aj), j= 1... (7-37) 
and for artificial vectors, 
а} = af” = (0, 1, ed. (7-38) 


The vector corresponding to xo is e;, and the vector corresponding to 
nı ÍS C2. ‘The rank of the matrix of the coefficients in (7-35) із т + 2 ` 
во that any basis matrix will be of order m + 2 and will be denoted by 


В; = (ei, bí)... bi) = BP, BI”, ..., Burs). (7-39) 


It will be assumed that the first column of By always contains е. In 
Phase I, the second column of B will always contain ез. A basic solution 
to (7-35) is feasible if all variables except zo, z&44 аге non-negative in 
Phase I and all variables except zo are non-negative in Phase II. 

A basic feasible solution to (7-35) can be obtained by setting all variables 
to zero, except zo, т. and the artificial variables. The basis matrix is 


1 0 0...0 
01 lest 


Bi-|0.9 1501, (7-40) 
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and it is not an identity matrix. However, if we apply in two steps the 
procedure of Chapter 2 for computing the inverse by partitioning, we 
immediately see that the inverse is 


10 0... 0 
ш е) 1 0 | 0 

Bs! aliar булуы был jot : (7-41) 
: : JOD ҖЫ 


© 
© 
© 
m 


This is easily checked by showing that В; "By = In+2. 
If the right-hand side of (7-35) is denoted by b'? then the basic feasible 
solution xf? whose corresponding basis matrix is В, can be written 


xj? = By'b — [to zm, ai › TBM, (7-42) 
and when В» is given by (7—40), then 
m+1 
x? = l^ am. рэ bib, ..., һы} (7-43) 
ize 


Equations (7-41) and (7-43) yield the x?) and the Bz" needed to get 
_ Started, 


In Phase I, we can partition any By containing e; in the first column, 
ez in the second column, and m vectors af? as follows: 


1 0| e 
B; = |0 1 = : 
0|B 


where B is the corresponding basis for Ax 


Zj — с; for Phase I, that is, 5 — ch. 

To determine the vector to enter the basis in Phase I 
Product of the second row of By? with each legitimate a® not in the basis. 
The af” yielding the smallest such number is the Eolo to enter the basis 
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provided the smallest number is negative. If we define 
у” = Bz a? = [Yon Vin - 0+1, (7-44) 


it is clear that since B7" appears in the lower right-hand corner of Bz", the 
last m components of y? are the y; for the corresponding basic solution to 
Ax — b. Similarly, the last m components of х{2 are the tg: Because of 
our previous change in the subscripts of the aij, the quantities z5,;..1, 
Yi41,j in (7-42) and (7-44) correspond to tgi, yi; for the corresponding 
solution to Ax = b. 

In Phase I, the first two columns never leave the basis. Hence the vector 
to be removed is determined from 


“ar — min f2 vir > 0}, Фо amity у (0245) 
+ 


Yrk i 
where the xpi, yi, are those defined by (7—42) and (7-44). 

We are now ready to perform the transformation to the new basic 
solution. This can be done exactly as in Standard Form I, and hence we 
shall not repeat it. Phase I terminates when Та = 0 or when all the 
zj — с; for Phase I are non-negative. 

For Phase II, we can imagine B5 partitioned as 


В; = l Bah (7-46) 
0 B 


where B, is a basis matrix of order m + 1 for the set of constraints which 
includes the constraint maintaining the artificial vectors at a zero level. 
From this it immediately follows that the first row of Bz Vis (1, cP Br"), 
and c? contains the m + 1 prices of the vectors in Bi. Therefore, the 
scalar product of the first row of Bz- ! with any legitimate vector aj? gives 
2; — cj for Phase II. Hence the scalar product of the first row of By” H 
with the a(? yields the set of numbers which determines the vector to 
enter the basis. 

In Phase II, хат is treated like any other artificial variable; it can 
be removed from the basic solution. Only zo must always remain in the 
basic solution. However, we are sure that there will always be at least one 
artificial vector in Ву because it is not possible to have an (m + 2)-dimen- 
sional basis without one. Haying decided that af” willenter the basis, 


we compute 
y = В;'а{” = [yor Vie - - -: Vm4A k]- 


The vector to leave the basis is then found from 


E£Br min EÊ yy, i= 
E min {ге ya > 0) i bise Ha (7-47) 
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The tableau for Standard Form II is the same as for Standard Form I 
except that there are an additional row and column. Although the second 
column of By" is always е» in Phase I, it is included in the tableau since it 
may change in Phase II. It should always be remembered in Phase II that 
there is a 1 which is the first component of the first row of Bz" that does 
not appear in the tableau since the first column of Bz' is not included. 

It may appear at first glance that more computations are required if 
Standard Form II of the revised simplex method is used than if (7-32) 
were used for Phase I, and (7-33) for Phase II only when one or more 
artificial vectors appear in the basis at a zero level at the end of Phase I Ў 
i.e., it seems unnecessary to carry along the extra equation in Phase I and 
to use the additional constraint in Phase II unless it is required. However, 
this is not correct. If the basis of order m + 2 were not used in Phase I, - 
then, at the end of Phase I, computations would have to be performed to 
find the first row (0, cB; `) for the basis inverse to be used in the initial 
iteration of Phase II; m or more iterations would be necessary in Phase I 
before the computations would equal those required to make the conver- 
sion. If no artificial vectors appear in the basis at the end of Phase I, 
no computations are required to transform the second row of By! in 
Phase II, since z,4.4 will never leave the basis and the second row will 
remain ej. Finally, using the same format in both phases simplifies the 
coding of the problem for a digital computer. 


7-6 Initial identity matrix for Phase I. The line of 1’s in the second row 
of the initial basis (7-40) did not bother us since it was very easy to invert 
the matrix by partitioning. Dantzig ef al. found it of advantage to begin 
the solution on a digital computer, using an exaet identity matrix as à 
basis matrix for a reason to be explained in Section 7-9. Hence the row of 
I's in (7-40) had to be removed; "This can be easily done: Suppose that in 
(7-35) we subtract from the second row the sum of rows 3 through m + 2, 
i.e., we consider 


This yields a new equation, 2 


mad 
ты — XOT c3 e 


(7-48) 
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Let us use (7-48) for the second equation. The new system can be written 


zot aeui +++ + aontn = 0, 
auzi +e Hint Pipi = b, 
azizi + +++ + antn + tn42 = be, (7-49) 
Om--1,121 E °? F dmn Ханн Ва, 
where 
ml ml 
E зуе O A А E 
=2 =2 


We now apply the revised simplex method to the system (7-49). It will 
be noted that the initial basis matrix В» is a strict identity matrix In 2. 
To convince ourselves that (7-49) is equivalent to (7-35), it is only neces- 
sary to show that maximizing z,4.1 [given by the second row of (7-49)] 
drives all artificial variables to zero, provided they can be driven to zero. 
First note that when each artificial variable is zero, then 277. Qijtj = 
b; (i = 2,...,m + 1), and hence, 


SITUE, UR 


i-2 i2 i2 


ог tn41 = 0. Furthermore, since each artificial variable is positive, 


п ml ml 
4$ outs > H S s 
jul ind i=? 
and z,4; < 0 always. Finally, if any artificial variable is positive, 
nı < 0. Therefore, if maxz,4; = 0, maximizing x4, in (7-49) 
drives all artificial variables to zero. 

The details of applying the revised simplex method to (7-49) are pre- 
cisely what has been already described, provided we make the following 
definitions: 

aj? = (ej, —1а,ај; БӘ = (0, —1b, b]. (7-50) 


7-7 An example. Using the revised simplex method, let us solve the 
following linear programming problem: 
2r, + 522 2 6, 
t+ za 2 2, 
21,22 > 0, 
min 2 = x; + 2z». 


(7-51) 


212 THE REVISED SIMPLEX METHOD [cHAP. 7 


This problem can be converted to 


2x, + бх — хз + ха = 6, 
t+ 3 — 24 d аз = 2, 
T1, 22, 73, Z4, Та, Taz > 0, 
max то = — zı — 203. 
Since we had to add artificial variables, we must use Standard Form II 


of the revised simplex method. In this form, the problem becomes (if we 
Write Ха = 26, Tag = 27) 


tot Tı + 225 =0, 
zs t-ze +} 2; = 0, 

2x1 + 522 — тз + 26 = 6, 
Tit za = 4 T £422. 


In Phase I, we maximize z5 and in Phase II, we maximize то. The vectors 
are 


aP = [1,0,2,1], a = [2,0,5,1], а = (0,0, —1, 0), 
аф  —[00,0,—1, а? =е„ а = (0,1, 1,0), 
aP —[0101, b” = (00,62) 


The inverse of the initial basis is 


EO АЧ EO 
вул |0 roi. 
Oe ulna 
О оао а 


and the initial basic feasible solution is 
ХР = [zo 25, тв, тт] = (0, —8, 6, 2]. 


The initial tableau is shown in Table (7-2). The first column of By? is 
not included because it is always ej. The y-column is not filled in until 
the vector to enter the basis is determined, 

In Phase I, we determine the vector to enter the basis by finding the 
smallest 27 — cf < 0, where zt — сў is the scalar product of the 
second row of Вх! with a). At the first iteration 
—2-12-442-de-—5—1- | 


—6, 23 — cf = 1,28 — cf = 1. 
Hence af" enters the basis. н am 


Now we compute у? = By'a®. This 


we have zf — ct = - 


تت —— 
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TABLE 7-2 — TABLEAU 1 


Variables 
in ge Be BY x?) yp 
Basis 
то 0 0 0 0 2 
25 1 —1 —1 —8 —6 
26 0 1 0 6 5 
27 0 0 1 2 1 


result appears in the yf’-column of Table 7-2. The vector to leave ће 
basis is determined from 


min ($, $) = $. 


Thus b$, the third column of Ba, is replaced by a$. 
A new tableau for the new basic feasible solution must be found. If we 


use the vector transformation formula, then 
«e = (—2, $,4 — 1, —3] = [—0.40, 1.20, —0.80, —0.20], 


and 
BS = BP + gua; P = х Hapa. 
For example, 
82 = [0, —1, 1, 0] + 1[—0.40, 1.20, —0.80, —0.20] 
= [—0.40, 0.20, 0.20, —0.20], 
and 
$9? = [0, =8, 6, 2] + 6[—0.40, 1.20, —0.80, —0.20] 
= [—2.40, —0.80, 1.20, 0.80]. 


The new tableau is presented in Table 7-3. 
At this stage, т» is still not zero; hence the end of Phase T has not been 


reached. Forming the scalar ч of the second row of B3 in Table 7-3 
with each a) not in the basis, we find 


zt — ci = —0.6, zš — с} = —0.20, 2 — с{ = 1. 
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TABLE 7-3 — TABLEAU 2 


Variables 
in | P| вр | op | xp | ур 
Basis 
хо 0 —0.40 0 —2.40 0.20 
25 1 0.20 —1 —0.80 | —0.60 
E21 0 0.20 0 1.20 0.40 
27 0 —0.20 1 0.80 0.60 


Therefore af? enters at the next iteration. Then y? is computed. The 
vector to be removed is the last column of the basis. The new tableau is 
given in Table 7-4. Here zo = 0; hence all artificial variables are zero, 
and Phase I is ended, 

To initiate Phase II, we take the scalar product of the first row of By! 
in Table 7-4 with each af? not in the basis. This gives 


23 — сз = 0.333, 24177 €4,—.0.333. 


Both of these values are positive, and consequently the solution is optimal. 
No iterations need to be made in Phase II. We see that minz — —max Zo 
= 2.667. The y{?-column in the last tableau is not filled in since no more 
iterations are to be made. In this example, we did not replace the second 
row УІ Ta; = 0 by (7-48). We could have done this, but for hand 


TABLE 7-4 — TABLEAU 3 
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computations it is not necessary. The reader may easily check the cor- 
rectness of the solution by solving the problem graphically. 


7-8 Comparison of the simplex and revised simplex methods. The 
revised simplex method has not met with wide acceptance for hand com- 
putations, although the computational effort is normally not much greater, 
and can even be less, than that required by the standard simplex method. 
Probably one of the reasons is the fact that side computations must always 
be made on scratch paper, while a tableau of the original vectors has to be 
available at the same time. These requirements can lead to more computa- 
tional mistakes than might be made with the simplex method. 

If a digital computer is used, the above objections are of no importance. 
In this case, the revised simplex method has some real advantages over the 
simplex method. Let us first make a rough estimate of the total number of 
computations required to move from one iteration to the next, using 
(1) the simplex method and (2) the revised simplex method.* For the 
simplex method, n — m + 1 columns must be transformed. The m 
columns in the new basis form an identity matrix in the tableau, and no 
arithmetic operations are necessary to obtain them. The extra column is 
the column yo = хв. We have to transform m + 1 elements in each 
column (y; and ум; = 2; — cj. A total of (m + 1)(n — m + 1) 
elements undergo transformation. Once we have Yix/Yrk, the only multi- 
plication ів yz;(yix/yrx). Thus, in changing tableaux, we essentially need 
(m+ 1)(n — m 4- 1) = nm — m? + n multiplications. Furthermore, 
m divisions are needed to obtain the y;./yr%. The total number of opera- 
tions is about nm — m? 4- n +m. For n = 20, m = 5, this would 
amount to 100 operations. 

Consider now the revised simplex method. Thete are m + 1 columns 
with m + 1 elements in each column, which must be transformed. Again, 
one multiplication is required for each. This results in about m? + 2m op- 
erations. Computation of then — m values of z; — с; requires (n — m)m 
operations since m multiplications are needed to find each 2; — су. Also, 
m? multiplications are required to compute the last m components of yf, 
and m divisions to determine y;;/yr;. Thus, the total number of operations 
for the revised simplex method is approximately nm + m? + 3m. For 
n = 20, т = 5, this amounts to 140 operations—considerably more than 
for the simplex method. 

While on a priori grounds the revised simplex method might seem to 
require more computations for changing bases than the simplex method, 


*'This estimate will be made for Standard Form I of the revised simplex 
method. Standard Form II requires slightly more computational effort than 
Standard Form I, but normally the additional amount is not significant. 
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it can turn out that it will actually require less. Suppose that we start out 
with a matrix which has a large number of zeros. This situation occurs 
rather frequently. In the simplex method, as changes are made from one 
tableau to the next, the zeros are soon replaced by y;; which are different 
Írom zero. Hence, in moving from one tableau to the next, we soon find 
it necessary to carry out all nm — m? + n + m operations. With the 
revised simplex method, however, all the z; — cj, yf? are computed by 
‚ means of the original af". Thus the zeros are preserved, and the total num- 
ber of operations required can be considerably less than nm + m? + 3m. 

Another advantage of the revised simplex method is that fewer entries 
need to be made in each tableau. This aspect is not too important for 
hand computations, but may be important when the problern is solved on 
a digital computer. For the simplex method, essentially (m + 1)(n + 1) 
elements must be written in each tableau. With the revised simplex 
method, only (m + 1)(m + 2) elements need be recorded. If n is con- 
siderably larger than m, a great deal less writing is required in the revised 
simplex method. A computer would normally record the tableaux for a 
large problem on magnetic tape. This form of writing is relatively time-con- 
suming, and consequently, it is important to reduce (whenever possible) 
the amount of writing required. 

A final advantage of the revised simplex method lies in the control of 
rounding-off errors which occur when a digital computer is used. Since a 
digital computer has only a fixed number of digits, every multiplication or 
division requires a rounding-off operation. After thousands of operations, 
the errors due to rounding off can become quite significant. Rounding-off 
errors are one of the factors that limit the size of a problem which can be 
solved on a digital computer. For a problem with a large percentage of 
zeros, the revised simplex method requires, at each iteration, considerably 
fewer multiplications involving nonzero elements than does the simplex 
method where zeros soon disappear. This reduction in the number of 
operations implies a corresponding reduction in the rounding-off errors. 

The revised simplex method allows rounding-off errors to be controlled 
in another way. The basis is transformed at each iteration by the usual 
transformation formula for the simplex method. This means that the new 
basis inverse is found by multiplying the current inverse by an E-matrix. 
After m iterations, more operations will have been performed to transform 
the inverse than if the matrix had been inverted directly. Thus, if the code 
for solving a linear programming problem has a built-in provision for 
directly inverting the basis matrix at any point, rounding-off errors can 
be kept within bounds. It is theoretically possible to prevent rounding- 
off errors from growing much above those that would result after m 
o d Nd oni it is quite time-consuming to invert the basis 

› never do this unless the problem required con- 
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siderably more than m iterations. Some judgment is required by the person 
solving the problem to know whether it is advisable to re-invert the basis. 
It might be noted that a basis-inversion routine could be used with the 
simplex method, but it would not be nearly so useful as it is with the 
revised simplex method because, after re-inversion, a new tableau would 
have to be computed by means of the new basis. 

In addition to its practical value for machine computations, the revised 
simplex method is also of theoretical interest. It will be of help in the 
discussion of duality in Chapter 8. 


7-9 The product form of the inverse. When the revised simplex method 
is used for hand computations, the following transformations are made at 
each iteration:* 


B- EB} &p = Eka! (1-52) 


The new inverse and new basic solution are computed by means of (7-52). 
When problems are solved on digital computers, the inverse is usually 
stored on tape. Then there are advantages to be gained if one never spe- 
cifically computes the inverse at all, but instead represents it in its product, 
form, ie., as the product of E-matrices, After p iterations the inverse will 
be represented as the product of p, E-matrices. It is really unnecessary to 
record the entire E-matrix in computer storage, Only the column т] 
and an index telling which column of E is s need be recorded since E 
differs from the identity matrix only because of the column containing э]. 
Thus recording the n-vectors, along with an index indicating the proper 
column of the respective E-matrices which contain n, is equivalent to 
having the inyerse of the basis. Using the product form of the inverse 
will reduce the time required to write the new inverse on tape since only 
a single column has to be recorded at each iteration. 
After p iterations, B^! will have the form 


B^! = E Ep ~. . Bı. ˆ (7-53) 


Assume that э" appears in column r of E,. Since B7! is never computed 
explicitly, we must show how to determine y, and the 2; — cj, using only 
Ле q'-vectors. Consider first yy. Now 

ух = Boa, = E,E,_1--- Еак (7-54) 


Let 
qu ERUNT Eiap oe a1, Ир. (7-55) 


* We here drop the subscripts and superseripts 1, 2 since the discussion applies 
both to Standard Form I and Standard Form I1. 
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Then ук = а? and о" = E,o-' ог, in component form, 


a = аб + mor, it,  s=1,...,7, 


ат = war, (7-56) 
where œ? = az. 

We have seen how to compute yx by means of the g*-vectors. It remains 
to show how the z; — c; can be computed when the inverse is kept in the 
product form. Let us suppose that we are in Phase II so that zj 2618 
obtained by taking the scalar product of the first row of the inverse with 
activity vector aj. Denote the first row of B^! by 8°. Therefore, 


B? = eiB7' = eE Ep; “Bj (1-57) 
and if ү = e/E,E,_,--- Epp {= 1,...,p, then Y! = YCÓE, п 
and 
Ут NI HS ke NN E HA (7-58) 


where ү? = ej, 4? = 80, Finally z; — су = B^aj. Precisely the same 
transformation formulas can be used in Phase I, except that ү? = ef. It 
will be observed that to use the product form of the inverse, it is necessary 
that the initial basis matrix be an identity matrix. This is why it is neces- 
sary in Standard Form II о make the transformation discussed in Section 
7-6 in order to obtain an identity matrix if the inverse of the matrix is 
used in its product form. It is also true that to use Standard Form I, along 
with the product of the inverse, Eq. (7-17) must be an identity matrix, 
ie, cg = 0. No transformation like that discussed in Section 7-6 can be 
used here to convert cp to zero (why?). Unless there is an identity matrix 
of order m + 1 in (7-5), Standard Form II must be used. 
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PROBLEMS 


7-1 Solve the following linear programming problem by the revised simplex 
method : 
8x1 + 472 £ 6, 
6x1 z2 < 3, 
ame > 0, 
maxz = 221 + 22. 


7-2, Solve the following lincar programming problem by the revised simplex 
method: 
zip 2n 2 7, 
4x, + z2 > 6, 
21,22 2 0; 
‘minz = zi + 22. 


1-3. Solve the following linear programming problem by the revised simplex 
method: 1 
221 — 822 + хз + 4х4 + 5z5 2 10, 
—83z1 + 412 + 523+ та + 925 2 5, 
zi 2ra 723 + Br. — 525 > 8, 
all z; > 0, 
maxz = 2д + 522 + Oza + tart 32s, 


7-4. Solve the following linear programming problem graphically and by the 
revised simplex method: 1 
215 4, 

13 — 21 2 0, 


x1, 22 2 0, 
maxz = 2x1 + 3z2. 


7-5. Solve the following problem graphically and by the, revised simplex 
method: { 
ж — 2 0, 

21, + 82; < —6, 

maxz = 872. 


Assume that the variables zı, 22 are unrestricted in sign. 

7-6. For the revised simplex method, discuss the types of computational 
errors that are self-correeting and the types that are not. Is it possible to use 
a sum-check column? [ 

1-1. Solve Problem 4-16 by the revised simplex method. 
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7-8. Show how the tableau format of the revised simplex method can be used 
at each iteration to select the vector to be removed from the basis in such a 
way that cycling will never occur. 

7-9. Discuss Standard Form II of the revised simplex method when it is not 
necessary to add an artificial variable to every constraint. Write the initial 
basis inverse corresponding to (7-41). 

7-10. In an article “A Two-Phase Method for the Simplex Tableau,” Opera- 
tions Research, 4, 4, pp. 443-47, 1956, Wagner suggests a version of the two- 
phase method for hand computations which is similar to the revised simplex 
method. If at the end of Phase I, there are some artificial vectors in the basis 
at a zero level, he leaves the (z; — cj)-row for Phase I in the tableau for Phase II, 
во that at the beginning of Phase II we have a basis of dimension m +- 1 
with 2, 1 in the basis at a zero level. A column for 2341, 90 = 654, is added 
to the tableau. However, in Phase П, Wagner does not allow =, 4.1 to be removed 
from the basis. He points out that if %n+41 can be removed, i.e., if Ym+1.k > 0, 
then there is another artificial vector with ya. = 0 that can be removed. Show 
that умы = —Y, Yik, Where i corresponds to the columns of the basis which 
contains an artificial vector at à zero level, Hence show that Wagner really does 
what we suggested for the two-phase method in Section 5-3, and that it is 
therefore unnecessary to introduce an additional row which only serves to point 
out an instance where, when i corresponds to some artificial vector, we have 
a Yik < 0. 

7-11. From Section 7-8 it will be observed that most of the computational 
effort in the revised simplex method is devoted to computing the z; — cj. 
Suppose that instead of using Eq. (7-18) to determine the vector to enter the 
basis, we simply choose the first vector for which z; — ej « 0. This could 
eliminate the computation of a large number of the 2; — cj. Discuss the ad- 
vantages and disadvantages of such a rule. 

7-12. Compare the number of arithmetic operations required when Eqs. (7-56) 
and (7-58) are used with that required when Eq. (7-52) is applied. 

7-13. Discuss in detail the difficulties that would be encountered if one at- 
tempted to use Eq. (7-41) for the initial basis inverse while at the same time 
using transformation equations of the form given in Eqs. (7-56), (7-58). 

7-14. Consider the Вә given by Eq. (7-46). Assume that no artificial vectors 
appear in the last m columns of Вә. Also assume that e2 is in column two. 
Then write By? in terms of B and Св, where B is the basis matrix for the cor- 
responding basic solution to Ax = b, 


CHAPTER 8 
DUALITY THEORY AND ITS RAMIFICATIONS 


“All of this of Pot and Potter—Tell me then 
who is the Potter, pray, and who the Pot? ii 


Edward Fitzgerald—The Rubayiat. 


8-1 Alternative formulations of linear programming problems. 'To 
solve a linear programming problem by the simplex method it was found 
convenient to convert the original set of constraint inequalities or equalities, 


r 
Dats = 2) e ЛУ (8-1) 
j=l 
to a set of simultaneous linear equations Ax = b, b > 0. A slightly dif- 
ferent formulation of the constraints will be found quite useful in treating 
the material discussed in this chapter. Instead of converting (8-1) to a set 
of equalities, we shall now convert (8-1) to a form such that each con- 
straint has an inequality sign (no strict equalities), with the additional 
property that the inequality sign is in the same direction for each con- 
straint. 
First let us observe that any equation 


DY aus = bi (8-2) 


qum 


in the original set of constraints (8-1) is equivalent to two inequalities: 


Dy titi S 0 
nr (8-3) 


В 
3a > б 


j=l 


The set of points lying on the hyperplane (8-2) is precisely the intersection 
of the two closed half-spaces (8-3). By use of (8-3) for (8-2) all constraints 
in (8-1) can be expressed as inequalities (no equations). If there were any 
equations in the original set of constraints (8-1), the new set of constraints 
containing only inequalities will have more constraints than (8-1) since 
two inequalities replace а single equation. 
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If, after converting all the equation constraints to inequality form, we 
multiply all those with a > sign by —1, all inequalities will have a < sign 
and can be written 


Duda S di i—1...,m-p, (8-4) 


j=! 


where p is the number of the original constraint equations. In matrix form, 
(8-4) can be written 


Dx, < d, D = |All: (8-5) 


Here it is not necessarily true that d > 0. The matrix D is (m + р) хт. 
We use the subscript r on x, (an r-component vector) to distinguish it from 
the x of Ax = b. 

In a similar way, we can multiply all the constraints with a < sign by 
—1 to obtain a > sign. Then the constraints become 


Gx, > g, (8-6) 


where G — —D and g = —d. 

We can now formulate the general linear programming problem in either 
of the two following forms: 

(a) Find an x, > 0 satisfying Dx, < d such that z = ¢,X, is maximized 
or minimized. 

(b) Find an x, > 0 satisfying Gx, > g such that z = C,X, is maximized 
or minimized. 

It should be noted that when the constraints are written in the form 
Ax = b, the matrix A will be of different dimensions than the matrices D, 
G. The matrix A is m X n, while the two matrices D, G are (m + p) X r. 
Also, x will usually contain some slack and/or surplus variables and will 
have more components than x,. 

In general, it will be clear from the context which formulation of the 
constraints we are considering, and hence the subscript r on x, will seldom 
be used. We shall simply abbreviate (a) and (b) by Dx « d,x > 0, max 
or minz = cx, and Gx > g,x > 0, max or min z = cx, respectively. 


e Dual linear programming problems. Given any linear programming 
problem: 


Dx < d, x 2 0, maxz — cx, А (8-7) 


there is another linear programming problem: 


Dw2c, w20, minZ= dw, (8-8) 
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which is called the dual of (8-7). We shall spend a considerable amount of 
time studying the relationship between problems (8-7) and (8-8) which, 
at first glance, may seem to be essentially unrelated. 

Note first of all that the dual of (8-7) contains the same constants dij, 
d;, cj as (8-7), but. in a rearranged order. The variables for (8-8) are, of 
course, different from those of (8-7). The primes in (8-8) denote, as usual, 
the transpose, If D is s X r so that x is an r-component vector, then D' is 
r X s, and w is an s-component vector. 

There is a good reason for formulating the constraints to the given 
linear programming problem (8-7) as a set of inequalities instead of as a 
set of equations. If the constraints to the given problem are written 
Ax = b, we shall later see that the variables of the dual problem are 
unrestricted in sign. If the variables for both problems are to be non- 
negative, the constraints must be written in the above form. 

On taking the transpose of the appropriate relations in (8-8), the dual 
of (8-7) can be written as 


wD > с, w' > 0, minZ — w'd. (8-9) 


It is helpful to attach a name to the given linear programming problem 
(8-7). We shall call it the primal problem. Both the primal and the dual 
problems can be conveniently represented at the same time in the following 


tableau format: 


215. yr 
Wi dii. -- diy dy 
eed |6 ri Fossa А (8-10) 
Wa |ds- -dsr d, 
IV 
(Crs 0r) 


The columns of D are the activity veetors for the primal, while the rows of 
D are the activity vectors for the dual problem. An abbreviated notation 


for (8-10) is 
24 


w (D) € d. 
IV 
c 


(8-11) 


In (8-10), the variable xj or ш; is written above or to the left of its corre- 
sponding column or row, respectively. 
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Equation (8-8) can be written 


(Dw € —c', w 2 0, max (Z*) — (—d’)w, (8-12) 
and £ 
minZ = —maxZ*. 


Now the dual problem “looks like” the primal problem. A very important 
observation ean be made at this point. If we consider (8-12) [or equiva- 
lently (8-8)] to be the given (primal) problem and write down its dual, we 
have from (8-12) and (8-7), (8-8), 


—Dx > —d, x 2 0, minz* — —cx, 
or | 

Dx < d, X20, max zi = cx, 
where 
min 2* = —maxz. 


This is precisely the original primal problem (8-7). For this reason we 
shall also define (8-7) to be the dual of (8-8). With this definition we see 
that the dual of thè dual is the primal. 

From this it immediately follows that instead of writing the primal 
problem in the form (8-7), we could have equally well used the formulation 


Gx > g, x20, min 2 = cx; (8-13) 
its dual is then 


Gwe’. w>o, тах 2 = g'w. (8-14) 


problems. If either problem in the set (8-7), (8-8) or (8-13), (8-14) is 
considered to be the primal, the other is its dual. 


In Chapters 11 and 13, it will be shown that the dual problem has some 
interesting economic interpretations, We shall begin the mathematical 
discussion by demonstrating that if the primal problem has an optimal 
solution; then the dual has also an optimal solution. In addition, we shall 
prove the even more amazing result that in Such a case max z — min Z for 
(8-7) and (8-8). Furthermore, it will be demonstrated that solving the 
primal problem by the simplex method also provides an optimal solution 
to the dual. Using this result, we shall find that, 
stances, the duality theory can be of considerable help in solving linear 
programming problems, 


m 
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ЕхАмрІ8: 1. Give the dual of the following linear programming prob- 


' lem: 
t | l5 < 1, z,,29 > 0, 
4 1| (22 —4 


max z = zı + 2re. 


This problem is given in the form of (8-7). The dual can be written down 
immediately; it is 


ЕЕ mes 


шїп Z = 3w; — 4ш;. 
2. Give the dual of the following linear programming problem (with 
non-negative dual variables): 
4r + 822+ тз = 6, 
zı za + 5ra = 4, (8-15) 
2,79, 23 2 0, 
maxz = 2z, + 322 + 23. 


We must first convert the problem either to the form of (8-7) or to that of 
(8-13). Let us write it in the form of (8-7). Then 


42, + 82 + 23 <. 6, 4 3 lllo: 6 
—42, — 3x2 — z3 S —6, lx —4 —3 —1|/ 22 < —6 
тү + 2т; + 5x4 S А, 1 2 5) | z3 4 

— zı — 223 — 523 < —4, —1 —2 —5 —4 


maxz = 2x1 + 3x2 + zs. 


The dual problem can now be written down. It is: 


4 —4 1 —1 2 
3 —3 2 —2|([wi we, wo, ws] > | 3| Wi Wa, w3, wa 2 0, 
1, ŞU S55 1 1 


min Z = бш — 602 + 4v3 — 404. 


If we had converted the primal problem to the form (8-13), we would have 
nonetheless obtained the same dual problem; however, all signs would 


have been reversed. 
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; „Ёё 1, 
. 1 e(t) 
С 


Figure 8-1 Figure 8-2 


According to its dictionary definition, dual means double. As applied to 
linear programming, duality implies that a double meaning can be attached 
to every linear programming problem. It can be thought of as a primal 
problem or as the dual of some other problem. Unless we are told, it is 
not possible to determine which interpretation someone is placing on a 
given problem. 

Nole: Duality is by no means a concept restricted to linear programming. 
It oceurs frequently in mathematics, economies, engineering, physics, and 
many other fields. 

As an example from electrical engineering, consider the following series 
RCL-network (Fig. 8-1). We assume that e is the voltage applied to the 
circuit, 7 is the current, and №, L, С are the resistance, inductance, and 
capacitance, respectively. The equation relating the current to the applied 
voltage at any given time is 


Gat eet | ie 
Lo + nit} fia = e(t). (8-16) 


If we interchange (e, i), and (L, C), and replace R by G = 1/R, we 
obtain 


, di 1 ] 
0 di tety fe dt = 10). (8-17) 


This is the equation representing the voltage in terms of the current for the 
parallel network of Fig, 8-2. Instead of-a voltage generator we have here 
a current generator. 

An equation of the form 


dz 
agi tbete fea = y0 (8-18) 


can be considered to describe a network of the type shown in Fig. 8-1 or 
of the type shown in Fig. 8-2, Unless someone tells us, we do not know 
which type of circuit it represents. Thus equation (8-18) has a dual inter- 
pretation. The network of Fig. 8-2 can be considered to be the dual of the 
network of Fig. 8-1 or, conversely, the network of Fig. 8-1 can be con- 
sidered to be the dual of the network of Fig. 8-2. 


8-2] DUAL LINEAR PROGRAMMING PROBLEMS 227 


Thus in electrie circuit theory, duality exists between series and parallel 
circuits, The conversions necessary to go from one cireuit.to its dual are 
listed below: 

Series Parallel 


e i 
R G 
L. С 
с L 


An example of duality which arises in economies appears in the Leontief 
input-output system; it, will be discussed in Chapter 13. This example is 
much closer to linear programming. If we imagine an economy to consist 
of n activities (n industries producing a single good), the Leontief input- 
output; model determines how much each industry should produce to meet 
a specified exogenous demand (consumer demand, foreign trade, ete.). 
Inter-industry transfers are automatically taken care of. With appropriate 
assumptions about the nature of the production process, the problem 
reduces to solving the following set of n simultaneous linear equations 
in n unknowns: 


(I— Ax — b, (8-19) 


where b is a vector of the net demands and A is а “technology matrix, " 
The components of x are the quantities of each good that must be produced. 

Another problem deals with determining the price of each good in terms 
of the value added in production. The prices p; are found as solutions to 


GLA par o (1- А)р= (8-20) 


where r is a vector of the values added and A’ is the transpose of A. 


„А system of equations 
Q- By =h (8-21) 


can be given a dual interpretation. It may be thought of as determining 
the quantities to be manufactured in one economy or the prices in another 
economy. Unless we are told, we do not know which interpretation some- 
one has in mind. 

The problem of determining the prices can be considered to be the dual 
of that of determining the quantities to be produced, or vice versa. Note 
that here, just as in linear programming, the dual problem involves the 
transposed matrix. ^ 

In the above examples, duality had a very important physical or eco- 
ain cases, an interesting economie interpre- 


nomie interpretation. In cert : 
dual linear programming problems. The 


tation can also be given to 
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duality, as in the Leontief models, exists between quantities and prices. 
This is discussed in Chapter 13. In Chapter 11, we shall also show that if 
the primal linear programming problem is one player's problem in a zero- 
sum, two-person game, then the dual is the other player’s problem. 


8-3 Fundamental properties of dual problems. In this section, we wish 
to show that if one of the pair of dual problems (8—7), (8-8) has an optimal 
solution, then the other also has an optimal solution. We shall also demon- 
strate how to find an optimal solution to the dual once we have deter- 
mined an optimal solution to the primal by use of the simplex method. 
We begin by proving the following statement: 

If x is any feasible solution to problem (8-7) and w is any feasible solution 
to problem (8-8), then 


cx X d'w, that is, z SZ. (8-22) 
To see this, observe that for any feasible solution to (8-7) we can write 
(Dx) S фо йул уй; 
and since w; > 0,7 = 1,..., s, for any feasible solution to (8-8), 
w;(Dx); < widi. 
Summing over 7, we have 


Zuw;(Dx); € Dud; or Ок < dw. (8-23) 


Similarly, since x > 0, it follows from (8-8), or equivalently from (8-9), 
that 
w'Dx > cx. (8-24) 


Combining (8-23) and (8-24), we obtain the result cx < d'w or z < 2, 

Next let us show that if 3 is a feasible solution to (8-7) and Ў is a feasible 
solution to (8-8) such that с® = d'w, then $ is an optimal solution to (8-7), 
and W 1з an optimal solution to (8-8). By assumption, 


ct = аҹ. (8-25) 
But for any feasible solution x, (8-22) holds or 


x X d'w = cf, . (8-26) 
and & is optimal. Similarly, for any feasible wW, 
dW = c& < d'w, (8-27) 


and therefore is optimal also. 
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Finally, let us show that if one of the set of problems (8-7), (8-8) has an 
optimal solution, then the other also has an optimal solution. (Recall that 
unbounded solutions are not called optimal.) We shall prove this by 
actually constructing an optimal solution to the dual from a given optimal 
solution to the primal. It suffices to show that if (8-7) has an optimal 
solution, then so does its dual (8-8). The fact that (8-7) has an optimal 
solution if (8-8) does, follows because, as we have seen, (8-8) can be con- 
verted to the form (8-7), and (8-7) can be converted to the form (8-8). 

To solve (8-7) by the simplex method, we first transform the constraint 
inequalities into a set of equations. Equation (8-7) can then be written 


Dx + Ix, = d, x20, x, > 0, maxz = cx, (8-28) 


where x, contains the slack variables. Assume that хв is an optimal basic 
feasible solution to (8-28); let B be the corresponding basis matrix and 
св contain the prices of the basic variables. Note that d is the same as in 
(8+7); hence it is not necessarily true that d 2 0. Imagine that an optimal 
solution has been found without making -each component of the require- 
ments vector non-negative (the reader who has worked Problem 4-24 
will have seen that it is very easy to get started on the simplex method even 
if some components of d are negative). 

Because xg is optimal, we have 2; — су 2 0 for all j and, in particular, 
for the vectors d; of D. Thus, 


CEB dj es ara’ jT) 
or 
cgB^! (di, ..., 4) 2 (С: , Cr), 


or 
cgB^!D > с. (8-29) 


If we write w' = свВ!, Eq. (8-9) shows that cpB™ is a solution to ће 


dual problem (8-8). 
When we consider the 23 — бу for the slack vectors of (8-28), we can 


immediately see that w = СЕВ! is feasible. Since the price associated 
with a slack variable is zero, the 25 — c; for the slack vectors are 


cgB e; — 0 2 0 


or 
cB 1(e1,..., 8) Z 0; 


hence 
cgB I > 0, and свв! 2 0. (8-30) 
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We have yet to show that w' = c5B-' is an optimal solution to the 
dual (8-8). Consider 


Z = w'd = cgB7'd = cgxg = maxz. (8-31) 


Here we have shown that w', xg are feasible solutions to (8-8), (8-7), 
respectively, for which z = Z; thus w’ is an optimal solution to (8-8). 

We have just obtained a very interesting result: If there is an optimal 
solution to (8-7), we have proved by construction that there is an optimal 
solution to its dual (8-8). Indeed, if B is the basis matrix for the primal 
problem corresponding to an optimal solution and св contains the prices 
in the basis, then an optimal solution to the dual is w = cgB^!. In 
addition, we have proved that max z — rain Z, i.e., the optimal values of. 
the objective functions for the two problems are equal. 

There is something familiar about the solution to the dual. Recall that 
if the revised simplex method is used to solve the primal, the first row of 
thé optimal tableau is (cgB-!, z). Hence, when the revised simplex 
method is used, the solution to the dual problem is contained in the first row 
of the optimal tableau for the primal. For simplicity, we gave above the 
first row for Standard Form I. In the event Standard Form II is used, 
the last s 4- 1 elements of the first row yield w' = c5B^', Z = г, 

If the primal problem is solved by the simplex method, it is also easy to 
find an optimal solution to the dual problem in an optimal tableau for the 
primal, From (8-30) we see that w — cB is found in the (z; — cj)-row 
under the columns corresponding to the slack vectors, that is, since for 
the slack vectors, z; — су = 2; = свВ- le; w; is found in the (z; — су)- 
position under the slack vector 1, etc. 

"Thus far we have been assuming that, prior to application of the simplex 
method, the primal problem has been cast into the form (8-7). It would 
be annoying if this procedure were always necessary in order to find the 
solution to the dual. Fortunately this is not the case. In the following 
section, we shall show that it is easy to obtain the solution to the dual 
when the primal problem is solved in the manner described in Chapter 4. 
It is immediately clear from the above arguments, however, that if 
Xa = B^'g is an optimal solution to (8-13), an optimal solution to the 
dual (8-14) is w = ¢,B™!. 

Without any additional explanations, we can see how the duality 
theory may, at times, be of assistance in actually solving a linear program- 
ming problem. If some given linear programming problem contains many 
constraints and only few variables, then the large number of constraints 
yields a large basis. When there are only few variables in the original set of 
constraint inequalities, most of the variables in this large basis will be slack 
or surplus variables. Let us consider the dual problem: The size of its basis 
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is that of the number of variables in the primal problem. Thus solving the 
dual provides all primal variables, and we need not solve a problem involv- 
ing a very large basis. An example will further clarify this point. 


ExAMPLE: Solve 


E21 X 6, 1 0 6 
тз $ 8) 0 1 

zij < 

г Ра € 7, or |l 1 E ari (8-32) 
T2 

3a, + 22 € 15, 3 1 15 

-—qa E 0 —1 1 
zj 22 > 0, 


maxz = 4r + 322. 


If we solved this problem directly, the basis matrix would be of order 5, 
while in reality there are only two variables, 21, zo, in which we are in- 
terested. 

The dual problem is 


1. 40, e ИЫН 
AES GU oe las zi! 3 


шу > 0, j21e35 
minZ — бш + 802 + 7ws + 1504 + ws. 


We shall solve the dual problem by the revised simplex method. An optimal 
solution to the primal can then be read off immediately from the optimal 
tableau for the dual. Since an identity matrix appears in the above con- 
straints, we can use Standard Form 1. When the dual problem is cast 


into this form, we have 
ye eg EMISIT 0 Ош 0 
0 53 ООН тече үн acd UU 
(aden leans ee } =! 0 —1 toy 3 


The vectors corresponding to the. 0; will be denoted by aj” and are 


numbered in order of their appearance. The tableaux for the solution to 
the dual problem are shown in Tables 8-1 through 8-3. The solution to 
the original problem is contained in the first row of Tableau 3. We read 
тү = 4,25 = 3, max 2 = 25. It should be noted that we did not convert 
the dual to a maximization problem; it was solved as a minimization 
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TABLE 8-1 — TABLEAU 1 


= 
Variables 
TAM Pir), Pap | we? Тур 
Basis 

wo 6 8 48 11 

wi 1 0 4 3 

w2 0 1 3 1 
L ET 


0.3333 


— n RN 


—0.3333 


Variables 
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problem. If тү, za are substituted into (8-32), it is seen that they do 
indeed yield a feasible solution with z = 25. It is optimal since z = Z. 
If we had solved the original problem directly, there would have been 
three slack vectors in the optimal basis at a nonzero level. The dual 
solution did not give us the values of these slack variables, but they can 
readily be computed once zı, тә are known. The correctness of the solution 
may be easily checked by solving the primal problem graphically. 


8-4 Other formulations of dual problems. Suppose that we have 
decided to solve some linear programming problem by means of the dual. 
After the dual has been obtained, it is not necessarily true that each com- 
ponent of the requirements vector is non-negative. Although we can get 
started on the simplex method even if some components of the require- 
ments vector are negative (by means of the procedure developed in Prob- 
lem 4-24), the usual procedure is to convert first the inequalities to a form 
such that the requirements vector is non-negative. One reason for doing 
this might be that the computer program to be used requires that the 
requirements vector be non-negative. We now wish to investigate what 
influence the transformation to a non-negative requirements vector for the 
dual has on obtaining an optimal solution to the primal. 

If we have the linear programming problem Dx < d, x > 0, max z = сх, 
and if one or more of the d; < 0, let us multiply the constraints with 
d; < 0 by —1. This gives a new set of constraints which has a non- 
negative requirements vector, but some constraints now have a > sign. 
After adding slack and surplus variables, we obtain the linear programming 
problem A(x, x,] = b, b > 0, [x,x,] 2 0, maxz = cx, where x, is à 
vector containing the slack and surplus variables. Corresponding to any 
basis matrix B for this problem, there is a basis matrix B for Dx -+ Ix, = d. 
Indeed B differs from B only in that the rows 7 of B for which d; < 0, are 
the negatives of the corresponding rows of B. When the basic feasible solu- 
tion of the given problem corresponding to B is optimal, then an optimal 
solution to the dual of the given problem will be ¢,B~'. Since we solve 
the problem with the non-negative requirements vector, cB ^! is obtained. 
We now wish to determine cB 1 from cB". 

If any row 7 of a matrix is multiplied by —1, then column 7 of the 
inverse is multiplied by —1. This follows immediately from the fact that 
B-!B — I. Therefore B-! differs from B^ only in that the columns û of 
В-! for which d; < 0 аге the negatives of the corresponding columns of 
B-1. Hence we see that the solution to the dual problem can be found from 

© cgB^! by multiplying by —1 those components i of this vector for which 
d; « 0. 

The above discussion shows that if we are solving some linear program- 

ming problem by means of the dual, and if we convert the requirements 


234 DUALITY THEORY AND ITS RAMIFICATIONS [снАР, 8 


vector of the dual into a non-negative one by multiplying the ith dual 
constraint by —1, then the ith primal variable computed from cB 7" of 
the dual problem will be nonpositive, and we must multiply it by —1 to 
obtain the correct answer. 


EXAMPLES: 1. Let us show by an example that changing the signs of 
row 7 in B changes the signs of column i in B~!, Let 


Ва 81h at]. 
L2 =! 2 
Suppose that we change the signs of the second row of B to yield a new 
matrix Bj. Then 


13 | B= 2 3], 
T7152 —11-2 


and the signs of the second column of B7! have been changed. 
2. Using the dual, solve the following linear programming problem: 
Ti 
T2 S 6, 
zı + 12 5 5, 
— ووا‎ € —1, 
Tı, 29 > 0, 
maxz = 3z, — 20. 
The dual is 
wi + шз 23, 
ш + wg — wy > +2, 
vj20 Foye 
min Z = 4w, + бш» + 5шз — шу. 
To solve this problem by the simplex method in the usual way we first 


multiply the second constraint by —1. After doing this and adding slack 
and surplus, we have: 


Vy ws — wg = 3, 

Wa — з + ws + we = 2, 

4520) dim dise, 
minZ = 401156054 5з — шу. 


1 This dual problem can be solved by inspection. The optimal revised 
simplex tableau is given in Table 8-4. To obtain the solution to the primal 


8-4] OTHER FORMULATIONS OF DUAL PROBLEMS 235 


TABLE 8-4 — TABLEAU 1 


we look at the first row of the tableau. Since we multiplied the second dual 
constraint by —1, we must multiply the second element of the first row 
by —1. This second element is indeed negative. Thus an optimal solution 
to the primal is 21 = 4, 22 = 1, maxz = 10. The correctness of this 
result can be easily checked by solving the primal problem graphically. 

We may cast the above results into a slightly different form. Essentially, 
what we have shown above is that multiplying a primal constraint by —1 
changes the sign of the corresponding dual variable. Suppose that we are 
given a linear programming problem of the following type: 


D dig; € ds = 1,...,Ё, 


jml 
P dyz dy dekkloag5 (8-33) 
j=l 

z; 20, all), 

maxz = cx. 


It follows that if D = ||d;j] and d = (dı, - . . , d), then the dual of this 


problem has the form 
Dw > с, (8-34) 


minZ = d'w, 
where the dual variables satisfy 


4x0 1... 


and 
ш <0 G=k+1,...,8) 
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To see this, multiply constraints k + 1, ..., s of (8-33) by —1, form the 
dual, and obtain 

k у В t 

2)%4*— У) wid >с, 


i=l i=k+1 


w;z0, i$—1.2.,5 


k Li 
minZ = J аа — У) dew 
i=l i=k+1 


where, of course, d' = (d;ı, ... , di). Introduce new variables w; — wj, 
i= l,..., уш = —w,i- k-1,...,5,and (8-34) results. 

Quite frequently computer codes for solving linear programming prob- 
lems make provisions only for maximizing the objective function. If we are 
solving some linear problem by means of the dual, let us see what happens 
if we have to convert a minimization problem to a maximization problem 
by changing the prices of the dual problem. Assume that B is a basis 
matrix for the dual corresponding to an optimal basie solution to the dual. 
Then the, solution to the primal is x” = dB", where dj contains the 
actual prices of the dual problem. To solve the dual problem the price of 
each variable is changed, and therefore the first row of an optimal revised 
simplex tableau would contain —d,B™', and we would need to multiply 
each component by —1 to find the correct solution to the primal. 

If a primal problem with an equality for one of its constraints is to be 
cast into the form of (8-7), the equation must be replaced by two in- 
equalities. This one constraint equation then yields two variables in the 
dual. The dual column corresponding to one of the dual variables is the 
negative of the other. Similarly, the price of one of these dual variables is 
Just the negative of the other. If we think back to the subject of unre- 
stricted variables, we see that the two dual variables can be replaced by a 
single unrestricted variable. The column corresponding to this unre- 
stricted variable would be that obtained if the original equation in the 
primal had not been converted to two inequalities. Thus we have the 
interesting result that if the ith constraint in the primal is an equality, then 
the ith dual variable is unrestricted in sign. 


The above paragraph has shown that if we write the primal problem 
in the form 


D dit; € d; :1—]2. 5k 

ј=1 

а — dy P up ль (8-35) 
fat 


a; > 0, a l...,r, max 2 = cx, 
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then the dual problem is 
D'w > с,  minZ = dw, (8-36) 


wi > 0, i = 1,..., w; unrestricted, SRB BL 


The first k variables of the dual are non-negative, and the remaining vari- 
ables are unrestricted in sign. 
A special case of (8-35) is presented by the primal problem 


Ax = b, x20, maxz = сх. (8-37) 


The dual must be 
Aw >с,  minZ = bw, (8-38) 


and all the dual variables are unrestricted in sign. Here we have found 
the dual of the standard formu ation of the linear programming problem 
developed in Chapter 4. The dual variables are all unrestricted. This is 
the reason why, at the beginning, we did not use (8-37) as the formulation 
of the primal. 

It is important to note that the presence of equality signs in some of the 
primal constraints in no way affects the inequality signs which appear in 
the dual. A change from inequalities to equalities in the primal affects the 
nature of the dual variables, not the nature of the inequalities signs in the 


dual constraints. 
ExaMPLE: Suppose that the primal problem is 


3z, +2r2 < 6, 
382, + 22 = 4, 
Tı, t2 2 0, 
maxz = 2 + 312. 


If the dual of this problem is written down directly from (8-36), we 


obtain 
Зшу + Зиг > 1, 
2ш + We = 3, 
wı > 0, 


minZ = 6w; + 4v». 


According to our discussion, шз should be an unrestricted variable. To 
demonstrate this, the equation in the original problem will be converted to 
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two inequalities. We find 
Зі + 2x2 < 6, 
9zi + T2 < 4, 
—3ry — T< —4. 
The dual problem is 
Зил + 8% — Зи% > 1, 
21+ we — ws > 3, 
wi, wo, ws > 0, 
min 2 = бш + 405 — 4v. 


If we write wa = w3 — 03, wı = wi, we see that the result is identical 
with the result obtained above without replacing the equation by two 
inequalities. As expected, wz is unrestricted. 


The reader can probably guess at this point that if some variable x; in 
the primal is unrestricted in sign, then the jth constraint of the dual problem 
will be a strict equality. This is correct and can be easily proved. Noté that 
in order to convert a primal problem with unrestricted variables into the 
form of (8-7), we replace x; by т; — 17. If d; is the vector corresponding 
to xj, then —d; is the vector corresponding to z4. Similarly, the prices 
corresponding to xj and z/' are с; and —cj, respectively. The columns 
corresponding to 2; and 2/ in the primal give the constraints dw > ĉj, 
—djw > —c;; these are equivalent to the single constraint djw = су. 
This single constraint corresponds to the single column d; in the primal, 
which must be associated with the unrestricted variable ту. Thus the 
following expressions are dual problems: ` 


Dx <d, maxz = cx, (8-39) 
with z; > 0,j = 1, ..., k, ху unrestricted for j = k+1,...,r, and 
" 
Уа > ej j= 1,...,k, 


i=l 
(8-40) 


А 
D dimi = cj Skt lar 
i=l 

w 20, 
minZ — d'w, 
It should be recognized that none of the dual formulations given in this 

Section is more general than the pair (8-7), (8-8). In fact, these new 


formulations were obtained from (8-7), (8-8). Furthermore, if хь = Bd, 
2 = CpXp is an optimal solution to any one of the formulations of the 
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primal considered in this section, then w’ = c5B-' is an optimal solution 
to the dual. Now, of course, it may not be true that all w; > 0. This 
depends on the nature of the primal. In Problem 8-9 the reader will be 
asked to show this in detail. 


8-5 Complementary slackness. Given a linear programming problem 
in thé form of (8-7), we know from Chapter 3 that if D is s X r, an optimal 
solution need not have more than s variables different from zero. If the 
problem (8-7) is solved by means of the dual, then an optimal solution to 
(8-7) is x’ = d5B-, where B is an optimal basis for the dual, and dz 
contains the dual prices in the optimal solution to the dual. Now B is 
an rth-order matrix, and therefore x' — d5B-' can apparently have r 
components different from zero. Normally, we would use the dual formula- 
tion only when r < s. In this case, the fact that we only find r < s values 
of z; merely means that the solution by the dual does not yield the values 
of any of the slack variables that would be added to (8-7) to obtain a set 
of equations, These slack variables may, of course, be easily computed 
from their defining equations. However, the solution to the dual does give 
some information about the slack variables. We shall prove below that if 
w; is in the optimal basic feasible solution to the dual, then for the corre- 
sponding optimal solution to the primal, x = 488), the ith primal 
constraint is satisfied as a strict equality, i.e., the ith primal slack variable 
18 zero, 

Suppose now that we solved (8-7) by using the dual and r > s. In 
this case, it would seem that more than s values of x could differ from zero. 
However, this is not true. The dual solution gives all values of the xj, those 
which are positive and those which are zero. Thus no more than s com- 
ponents of x’ = 4/87! will differ from zero. If r > s, the dual basis B 
must contain at least r — s surplus vectors. We shall prove the interesting 
result that if a surplus variable appears in column i of B, then z; of x’ = 
d5B- is zero. 

Let us now restate and prove the two theorems referred to in the above 
paragraphs. As usual we give the formulation in terms of the primal. The 


theorems are: 

(a) If a slack or surplus variable z, 4j which has been added to the ith primal 
constraint appears in an oplimal basic solution (with the basis matrix 
being B), then for the corresponding optimal solution to the dual w' = 
c5B-1, the ith dual variable is zero, that is, иӊ = 0. 

(b) If the variable x; appears in an optimal basic solution to the primal 
problem, then in the corresponding optimal solution to the dual, w' = 
cpB™', the jth dual constraint holds as a strict equality, that is, the dual 
slack or surplus variable w.4; = 0- 
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To prove (a) note that since 2,4; is in the optimal basic solution, and 
since the activity vector for z,,; is +e; and its price is zero, 


н — Cros = CBB (+e;) = -(cgB ); = ш; = 0. (8-41) 


If we let d; be the column vector corresponding to the ху, then the proof 
of (b) follows from the fact that z; — с; = 0 since 2; is in the optimal 
solution. It is only necessary to note that 


2j — 6) = свВ7'!4; = cj = w'd; — cj = 0, (8-42) 


so that w does satisfy the jth dual constraint as a strict equality. Observe 
that the proofs used hold for all formulations of the primal and are not 
restricted to the form (8-7). 

The results (a) and (b) are usually presented in a slightly different form. 
Consider (a) first. The modified form reads: 


(a) If a slack variable т, which has been added to the ith constraint of 
(8-7) is different from zero in any optimal solution to (8-7), then in (8-8) 
the ith variable w; is zero in every optimal solution. Conversely, if w; is 
different from zero in any optimal solution to (8-8), 24; = 0 in every 
optimal solution to (8-7). This can be summarized by saying that for any 
pair of optimal solutions to (8-7), (8-8), ойы = 0/4 = 1,9... , 8. 
Note that this holds for any optimal solutions to (8-7), (8-8), not merely 
for basic solutions. 


To prove this modified form of (a), we first convert the constraints to 
equations; this yields 
Dx + Ix, = d: 


Multiplying on the left by w’ > 0, we obtain 
w Dx + wx, = wd. (8-43) 


However, if [x, x,] is an optimal solution to the primal and w is an optimal 
solution to the dual, then according to (8-25) 


maxz — w'Dx — minZ — w'd; 


hence it follows from (8-43) that 


8 

wx, = > Vip = 0, ог Witry = 0, t= 1,...,8, (8-44) 
ici 

since wi > 0,z,,; 2 0. Therefore, if Try = 0, ш; = 0. Conversely, 

if w; = 0, z,,; = 0. This result is less general than (8-41) in that 

we must assume that 2,3; > 0 for at least one optimal solution to the 
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primal, to show that w; = 0 for any optimal solution to the dual. However, 
it is more general than (8-41) in the sense that it does not require the 
optimal solutions to be basie, and it holds for any form of optimal solutions 
rather than only for corresponding optimal solutions of the form хв = 
Bd, w = cgB~'. The modified version of (b) reads: 


(b) If x is any optimal solution to (8-7) and if w, contains the surplus 
variables for any optimal solution to (8-8), then 


x'w,=0, OF 240.) = 0,053 m DENE (8-45) ` 


The proof is precisely the same as that presented above if one begins with 
(8-8) instead of (8-7). 

The properties (a), (b) are sometimes referred to as complementary 
slackness. ‘To see why this name is used, let us include the non-negativity 
restrictions as constraints and write the primal and dual constraints as 


x 20, D'w 2 с’, 


(8-46) 
Dx < d, w 290. 


If the constraints are numbered in this order, and if for any pair of 
optimal solutions to (8-46), the kth constraint of the primal is a strict 
inequality, then the kth constraint of the dual is a strict equality. There- 
fore, the constraints which can be inequalities in the dual are complemen- 
tary to those which are strict inequalities in the primal. However, this 
does not imply that the kth dual constraint will be a strict inequality if 
the kth primal constraint is a strict equality (see Problem 8-11). Both 
the kth primal and the kth dual.constraints may hold as equalities. 

The remarks made in the preceding paragraph apply to any optimal 
solutions of the primal and dual problems. It is an interesting fact, that 
one ean prove that if the primal has an optimal solution, then there is one 
optimal solution to the primal and one optimal solution to the dual such 
that if ты 0, then ш; > 0, andif ту = 0, then ws; > 0; furthermore, 
if w; = 0, 2,44 > 0, and if waj = 0,2; > 0. This statement is stronger 
than the results obtained above. However, these latter results do not 
necessarily hold for all optimal solutions. The proof of the result given in 
this paragraph is left for Problem 8-13. 


8-6 Unbounded solution in the primal. We have shown that if the primal 
problem has an optimal solution, then so does the dual. If the primal 
problem has an unbounded solution, then the dual has either no solution 
or an unbounded solution. This follows because if the dual had an optimal 
solution, then the primal would also have an optimal solution since it is 
the dual of the dual. We shall now show that 7f the primal has an unbounded 
solution, the dual has no feasible solution. i 
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Let us assume that the primal problem has been cast into the form (8-7). 
We have shown previously that for any feasible solution w to the dual (8-8), 


maxz = max cx < d'w. (8-47) 


But, by assumption, max z = co, which means that there is no feasible w 
whose components are all finite. Hence there is no feasible solution to the 
dual. 

One must be careful not to conclude that if the primal has no solution, 
then the dual has an unbounded solution. It may be true that neither 
problem has a solution. ‘The following example illustrates this point. 


EXAMPLE: The linear programming problem, 


+з < 0, 
тт; > 0, 


тах г = Зі + 4х, 


has no solution since the constraints are inconsistent (see Fig. 8-3). 
Similarly, the dual problem below has no solution (see Fig. 8-4): 
ш — we 2 3, 
—w, + wa > 4, 
wı, wa > 0, 
minZ = —w,. 


8-7 Dual simplex algorithm.* The reader May have already noticed 
that for any linear programming problem, the Zj — cj are completely 
independent of the requirements vector. Consequently, the set of basic 


* This is the first time that we have used the word algorithm, although it 
appears frequently in linear programming. All numerical techniques for solving 
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solutions to Ax = b with z; — c; > 0 for all j depends only on the a; 
and the c; but not on b. In general, not 'every basie solution with all 
zj — cj = 0 will be feasible. However, any basic feasible solution with 
all z; — с; > 0 will be an optimal solution. 

The above observation presents an interesting possibility. If we could 
start with some basic, but not feasible, solution to a given linear program- 
ming problem which had all 2; — cj > 0 and move from this basic solution 
to another by changing one vector at a time in such a way that we kept 
all 2; — c; > 0, then, provided no basis had to be repeated, an optimal 
solution to the linear programming problem would be obtained in a finite 
number of steps. This is precisely what the dual simplex algorithm does. 
The fact that we maintain all the гу — су > 0 at each step and are not 
concerned about the feasibility of the basic solutions (i.e., about the 
requirements vector) suggests that the duality theory should be of help in 
developing such an algorithm. 

By examining the application of the simplex method to the dual of a 
linear programming problem, C. E. Lemke, a student of Charnes, dis- 
covered a new algorithm for solving the primal problem which meets the 
suggestions made in the above paragraph. Because it is not always easy to 
get started by means of this new algorithm, it does not have the general 
applicability of the usual simplex method. However, it can be used in 
certain cases to eliminate the necessity for a Phase I and the consequent 
introduction of artificial variables. It will also prove very helpful in treat- 
ing some of the problems to be discussed in Chapter 11. Lemke's algorithm 
has been called the dual simplex algorithm since the criteria for inserting 
and removing a vector are those for the dual rather than the primal 
problem. 

Let us assume that we have cast the given (primal) linear programming 
problem into the standard form for the application of the simplex method, 


ie, 


Ax=b, x20  maxz- cx (8-48) 
The dual of this problem is 
w'A > с, (8-49) 
minZ = w'b, 


and Section 8-4 has shown that the dual variables are unrestricted in sign. 


linear programming problems are called algorithms; thus, eg. the simplex 
method is called the simplex algorithm. Roughly speaking, an algorithm is just a 
prescription or specific routine for solving a given problem. A precise mathe- 
matical definition of an algorithm has been formulated only recently (1933) by 
Alonzo Church. He also showed that some problems are so complex that they 
have no algorithmic solutions. In 1936, Post and Turing independently formu- 
lated a very general theory of algorithms or routine processes. 
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For any feasible solution x to (8-48) and any solution w to (849), we 
have, by (8-48), i 
w'Ax = w'b, 
and by (8-49), 
w'Ax > cx. 


Consequently, for any feasible solution x to (8-48) and any solution w to 
(8-49), 
cx < w'b. (8-50) 


The derivation of the dual simplex algorithm will be first presented in 
the form of Lemke’s treatment, which is very neat and simple. Since 
his development does not indicate too well the intuitive motivation for 
the procedure, we shall, in addition, present a more straightforward 
and detailed treatment. 

As usual, A will be taken to be an m X n matrix. Note that the con- 
straints of (8-49) can be written 


WaPo, 1. (8-51) 


Suppose that we have obtained from A a basis matrix B = Oy be 
such that the solution w to the set of m simultaneous linear equations in 
m unknowns 


whi = ca, t=1,.:.,m, о w' -cgB^ 
is a solution to the dual, i.e., 
W'b; = cpi, i= 1,...,т, (8-52a) 
waj > Cj, (8-52b) 
' for every aj not in the basis. Since w' = c5B^, we can write 
c5gB^'aj — e; 2 0 allj, (8-53) 


or 2у — су > 0 for all j, which implies that the optimality criterion is 
satisfied for the primal problem. Then if 


xs = B-!b > 0, (8-54) 
we have an optimal solution to the primal and the dual since 
Сх = cpX5 = c5B-^ b = wb. (8-55) 


When we have a solution to the dual, w' = cgB-!, and it is not optimal, 
we are forced to conclude that one or more of the Zp, in (8-54) are negative. 
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Let us study the case where one or more xg; < 0. Denote the rows of 
B^! by 8’. It follows that 


B= р 063m cm (8-56) 
Consider any xg, < 0 which is given by 
хв = ВЪ. (8-57) 
We shall investigate the properties of a new vector W given by 
#' = w — 08", (8-58) 


Then 
Z = Wb = wb — 66" = Z — 6zp,. (8-59) 


Consequently, if 0 < 0, Z < Z since rg, < 0, and W will yield a smaller 
value of Z. If W can be made to satisfy the constraints of (8-49), we have 
a new solution to the dual with Z < Z7. 

Now observe that 


Wb; = wb; = 08", = св; — 08;,. (8-60) 
Thus, 

W'b; = Cai (i з r), (8-61a) 

Wp е 0р NER 1 8 0. (8-61b) 


For all a; not in the primal basis, 
Wa; = w'a; — 08'a;. (8-62) 
If B'a; > 0 for every a; not in the basis, then for any 0 < 0, 
Wa; > cj. (8-63) 


In such a situation, 0 can be made an arbitrarily large negative number, 
and we still have a solution to the dual. Thus Z can be made arbitrarily 
small, and the dual has an unbounded solution. From Section 8-6 we know 
that the primal has no feasible solution. 

Let us suppose that there is at least one aj such that 8'a; < 0. Note that 


В'а; = Uri, (8-64) 
where y,; is just the rth component of В-!а;. For all yr; < 0, we must 


have 
wa; — су > Or; 
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to obtain Y'a; > су. Therefore, we require that 


Ө > жа; — cj (8—65) 


since Jr; < 0. To decrease Z to the utmost, 0 must (a) be as large in 
absolute value as possible and (b) satisfy (8-65). The best we can do is to 
take 


festa cer yj « 0. 
i Yri 


Recall, however, that in terms of the primal notation, w'a; — су = 
г; — су > 0. Therefore, 


26 — С Zi — с; 
6 = HE — max 7 2 
Ук ј Yri 


| oy € 0. (8-66) 


We have found a new solution to the dual, given by 


Wb; = cpi, i*r, 


W'b, > CBr, 
Waj > сс fora; notin B and j > k, (8-67) 
Wa, = wa, — Oy, = c, 0< 0, 

Z=Z- 02 py. 


Ife <0 2< 7. Furthermore, note that the strict equality holds for the 
dual constraint involving ay. Since y, >< 0, ax, together with the b; (i = r), 
forms a new primal basis B in which column r of B has been replaced by ау. 

If £3 = B-'b still contains one or more negative components, we can 
repeat the entire process, Observe that the actual computatións do not 
require the dual problem at all. Although the changes to be made were 
developed from a consideration of the dual problem, we see that they can 
be expressed solely in terms of the primal. 

Hence, the new algorithm for the primal can be described as follows: 
We begin with a basic solution to the primal with one or more zgi < 0, 
but with z; — c; > O for all j. As in the simplex method, we move from 
this solution to an optimal solution by changing a single vector in the 
basis at each step. The criteria for the change of basis are: 

I. Vector to be removed: Choose 


Zar = Min tgi ZB; < 0. (8-68) 
i 


Column b, is removed from the basis, and Zz, is therefore driven to zero. 
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II. Vector to enter basis: The vector a; to enter the basis is determined 

from в «ЕСАБ ERE 20: (8-69) 
Vrk ra і Yri j yy E 

Originally, we chose any rg, < 0. The greatest decrease in Z will come 
when 625, is as large as possible. Instead of computing 6 for each xg; < 0, 
we simply choose the smallest xg; in (8-68) to determine the vector to 
leave the basis. This procedure is similar to choosing the smallest z; — c; 
rather than the smallest (rg,/y.)(z; — cj) in the simplex method. If 
there is a tie, any rule can be used to break it. 

The value of @ computed from (8-69) need not be negative; it may be 
zero. Then Z does not change, and we have a case of dual degeneracy. If 
at any iteration the maximum in Eq. (8-69) is not unique, then the | 
vector to enter is not uniquely determined. At the next step, there will be 
one or more 2; — c; = 0 for vectors not in the primal basis, and dual de- 
generacy will appear. This situation is best handled by using some arbi- 
trary rule for determining the vector to enter the basis. Under these 
circumstances, cycling can theoretically occur in the dual simplex algo- 
rithm. Problem 8-25 will ask the reader to show how cycling can 
rigorously be avoided. 

Note that in the dual simplex algorithm one first, determines the vector 
to leave the basis and then the vector to enter. This is the reverse of what 
is done in the simplex method. Also note that degeneracy and cycling 
appear in choosing the vector to enter the primal basis, whereas in the 
simplex method degeneracy causes difficulties in determining the vector 
to leave the basis. In the following section, we shall show that the ap- 
pearance of dual degeneracy in the primal implies the vanishing of one or 
more basic variables in the dual problem. j 

The reader may be wondering how we determine whether the primal 
problem has an unbounded solution when the dual simplex algorithm is 
used. If the primal has an unbounded solution, the dual problem has no 
solution, and consequently, we can never find a solution to the dual prob- 
lem. Hence if the primal has an unbounded solution, we can never find a 
basic solution to the primal with all z; = c; 2 0. 

We have shown that the dual simplex algorithm can be applied directly 
to the primal problem. In fact, at each iteration, we construct precisely 
the same type of tableau as in the simplex method. The formulas used to 
transform from one tableau to the next are indeed those used for the simplex 
method. The difference in the algorithms is manifested in the selection 
of the vectors to enter and leave the basis. 


8-8 Alternative derivation of the dual simplex algorithm. It is desirable 
to derive the dual simplex algorithm in a more straightforward way. We 
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shall do this by applying the simplex method directly to the dual problem. 
After adding surplus variables to the dual problem,* we can write it in 
the form used in applications of the simplex method, i.e., 


Aw — Iw, = c', 
w, > 0, (8-70) 


min Z = b'w. 


The w; of w are unrestricted in sign. 
Consider now any solution x (not necessarily feasible) to Ax = b and 
any solution w to the dual. Then 


XA'w — x'w, = x'c'. 
If it is true that 
x'w., = 0, (8-71) 
then 
Z = b'w = cx = 2, (8-72) 


Thus for any solution to the primal and any solution to the dual with 
x'w, = 0, it follows that the objective functions for both problems are 
equal. It will be noted that the dual simplex algorithm as described in the 
previous section does maintain x'w, = 0 at each stage. 

Let us apply the simplex method to (8-70). The constraint matrix 
(A’, —I)isn X (т + n). As usual, it will be assumed thatn > m. Any 
basis matrix B for the dual problem will be n X n. The columns of A’, 
are a’, the rows of A. We shall consider a basic feasible solution to the dual 
to be a basic solution with non-negative surplus variables. The variables 
ww; are unrestricted. Since there are only m columns in A’, there must always 
be at least n — m surplus variables in any basie solution whose basis 
matrix B is formed from the columns of (A', —I). 

Assume that we have found a basic feasible solution to the dual which 
contains the m vectors of A’. It will be helpful to study the structure of 
the basis matrix in some detail. Without loss in generality, we can assume 
that a", ..., a" are in the first m columns of B. Thelastn — m columns 
will contain surplus vectors of the form —e;. To be specific, take the sur- 
plus vectors to be @m41,..., €n. The rows of the dual problem can always 
be rearranged so that this will be true’ Then B is 


E: : б 
В = (а, ат, еа, че): 


* For clarity, we shall denote the surplus variables in this and the following 
sections by ш; rather than „уу. 
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The first m columns of B are A’. Therefore B can be written in terms of the 
columns of A as 


ai 050-7 0 
а 0 0 
Б = |а 9.0: 0]. (8-73) 


mp 
р. 
“А 
I 
t» 
© 
© 


ah 0 0s*- —I 


Of course, a,,..., Am in (8-73) may be any m columns of the original A 
[since we rearranged the rows of A' (columns of A) so that the surplus 
vectors would be those shown], provided that B yields a basic feasible 
solution to the dual. 

Since r(B) = n, the first m rows of B are linearly independent. There- 
fore, а, ..., am must be linearly independent. We immediately see that 
B = (a;,..., am) is a basis for the primal system. In terms of B, B can 
be written 


в- |р 0 |: а asl. (8-74) 
R —h-n 


We see that B, B yield solutions to the dual and primal, respectively, 
which have x'w, — 0. Whenever we have a basic feasible solution to the 
dual with a", . . . , a" in B, we can immediately find a basic solution to the 
primal by the method discussed above. The manner in which we obtained 
the primal from the dual basis clearly shows that when a; is in the primal 
basis, —e; is not in the dual basis, and conversely. Consequently, x; = 0 
when w,; # 0, and w,; = 0 when 2; # 0. This means that (8-71) holds, 
andz = Z for the corresponding solutions to the primal and dual problems. 

Having obtained the basic feasible solution to the dual discussed above, 
we now wish to know whether it is optimal or whether it can be improved. 
When B is of the form (8-73), the prices of the variables in the dual basis 
will be the b; for the first m columns and 0 for the remaining columns. 
Denote the price vector by bz = (b’, 0). Note that the only vectors for 
the dual problem not in the dual basis are surplus vectors. Write 


Z; = bgB (—e) ?=1,...,т. (8-75) 


The basic feasible solution to the dual will be optimal if Z; € 0 for the 
surplus vectors not in the dual basis. This follows from the optimality 
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criterion of the simplex method applied to a minimization problem (recall 
that the price of a surplus variable is zero). 
Using (8-74), we can write 


Bel de i: 6-76) 
RBIS (=F 
Therefore, 
иу —1 
а-о) je = 0-60 = з. (8-77) 
R@B) —I : 


Thus, if all ар; > 0, we have an optimal solution to the dual, and Xp is 
an optimal solution to the primal, as expected. 

If one or more Z; > 0, we know that Z can be decreased (or that at 
least a new basis with the same value of Z can be obtained), Using the 
simplex criterion, we choose the vector to enter the dual basis from 


2. = maxZ, 2; >0, (8-78) 


or, in terms of the z5;, we determine the vector to be removed from the 
primal basis, using 
Br = min ni, Tp; < 0. (8-79) 


"This js the result obtained in the previous section in a less Straightforward 
way. According to (8-78) or (8-79), —e, enters the dual basis at the next 
iteration. It follows that the primal’ variable Corresponding to хр, is 
thereby driven to zero, 

Before determining which vector should leave the dual basis, let us 
compute the basic feasible solution to the dual wa; we have 


wg = Be = [> ‘| | й (8-80) 
RB)" I| ler 


and cr contains the prices not in the primal basis B. Also the first m 
components of wg are w. For these first m components, 


w= (B) o. w= c5B-!; (8-81) 


this is the same result as that obtained in the previous section. 
The last n — m components of Wğ аге 


RB’) es — c, 
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or, taking the transpose, we have 
cpB IR’ — cg. 
In terms of components, this is 
шу = cgB ay — су = 2; — су. (8-82) 


Thus the surplus variable w,; in the dual basis has the value z; — c; 
for the primal variable z; not in the primal basis. Since z; — c; — 0 for 
the vectors in the primal basis, and since —e; is not in B if a; is in B, we 
can also write w,; = zj — cj for the surplus variables not in the dual 
basis. Hence, 

wef S27 0) PSY ym. (8-83) 


We can now turn to the subject of deciding which vector should leave 
the dual basis. We never want to remove a',..., a" because if we do, we 
shall not be able to determine a corresponding basis for the primal problem. 
We know that if r(A) = m, an optimal solution to the primal will have m 
of the aj in the primal basis, and therefore at least m of the dual con- 
straints will be striet equalities. If we write 


у' = В7(—е,), (8-84) 


n- муж 
y- ke é J (—е) = i li |: (8-85) 
“I 


then 


R(B)-' R(B')-'ef 


where e* is a vector containing the first m components of e,. It follows 
from our previous arrangement that the last n — m components of any 
—e, not in the basis are zero. 

The vector to be removed from the surplus vectors in the basis is, as 
usual, determined from 


Us min Н, > 0p — Fom+t,...,n (8-80) 
Vk j (Yi 


We can ignore the first m components of wg because they are unrestricted 
in sign, and we do not want to remove any of the first m columns. Let us 
evaluate more explicitly the last n — m components of у". Recall that 
y; for the corresponding basic solution to the primal is given by у; = B^'a; 
or у! = a/(B')-!. However, aj is one of the rows of R. Thus R(B)-'isa 
matrix of the y/ for vectors not in the primal basis. Also yje? = y,;. Thus 
we see that the last n — m components of у" are the —y;;. In particular, 
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Vj = —Yr;. One must be careful to note the interchange of subscripts; 
yj is the jth component of y' for the dual, and Yr; is the rth component of 
y; for the primal. 
"Thus, in terms of the primal, the criterion for the vector to be removed 
from the dual basis is, from (8-86), 
£k — Ck iS — CF 


= min > 0. 
—Yrk le a Sid 


Zk — Ck 27—00) 
Yrk jo Wn 


д Yri < 0, (8-87) 


which is the result obtained in the preceding section. Of course, if y,; > 0 
for all vectors not in the primal basis, the dual has an unbounded solution, 
and the primal has no solution. 


8-9 Initial solution for dual simplex algorithm. The dual simplex algo- 
rithm will be used to solve a linear programming problem only if it reduces 


algorithm has other important applications, which will be discussed later. 
In these cases, we always have an initial solution for the dual simplex 
algorithm. 

In general, it is not an easy task to find an initial basic solution to the 
primal with all 2305103220: Equivalently, this implies that it can be 
difficult to find a basic feasible solution to the dual. In the worst case, 
we may have to add л artificial vectors to the dual problem. Even after 
determining a feasible solution to the dual in this manner, we may not 
find a',..., a" in the basis, and more work would be required to obtain a 
basic solution to the primal with all z; — с; > 0. Clearly, such a procedure 
could be considerably more time-consuming than the direct application of 

. the two-phase method to the primal, 

There is one particular case where one can easily find a basic solution to 
the primal problem with all 2; — cj > 0. Suppose that a surplus variable 
has been added to each constraint, Also assume that c; < 0 for every j. 
If the initial basis matrix is B = —L then хв —.—b, and 06 
=c; Z 0. This follows since the columns of —I are surplus vectors and 
their price is 0. Here we have a basic solution with all z; — cj = 0, and 
the dual simplex algorithm can now be applied. 

At least two other methods have been Suggested, one by Lemke and one 
by Dantzig, for getting started on the dual simplex algorithm. "They re- 
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quire a good deal of work and hence seem to offer no advántage over the 
addition of artificial variables to the primal. The details of these two 
methods are studied in Problems 8-23 and 8-24. 


8-10 The dual simplex algorithm; an example. Consider the following 
linear programming problem: 
m+ 222 1, 
2x, + 329 > 2, (8-88) 
21,29 2 0, 


max 2 = —3110— 22. 
After conversion to the standard form for the simplex method, we have 


zip od. — 23 = 1, 
2z, + 8х; — z4 = 2, 
maxz = —3z, — Т2. 


If the problem were to be solved by the standard simplex method, we 
would need to introduce two artificial vectors. Suppose instead that we 
consider the basis which contains 23, 24, that is, 


pst per] 
0771 0 —1 
For this basis, xg = [—1, —2]. Furthermore 2; — e; = —су = 3, 
гв — cy = 1. Here we have a basic feasible solution with all z; — c; > 0. 
Hence the dual simplex method can be used, and artificial vectors are not 
needed. 

The initial tableau is shown in Table 8-5. Note that y; = —a; since 
B = —I In the dual simplex method we first choose the vector to leave 
the basis, using (8-68); 222 = —2 is the most negative tai, and therefore 
the second column of B will be removed. It may now be helpful to encircle 
the second row of tableau 1, as shown. 

The vector to enter is determined from (8-69): We select all the у»; < 0, 
that is, all the yo; < 0 (the у»; in the encircled row). In our particular 
case, both ya; and yo» < 0. Then we divide the z; — cj with y,j < 0 by 
у; and select the largest number. Hence, 


2 — Ch E 2-9 1 1. 
Yrk 


Therefore, az enters the basis. We can now encircle column аз. 


254 DUALITY THEORY AND ITS RAMIFICATIONS [снАР. 8 


TABLE 8-5 — TABLEAU 1 


The transformation to the new tableau is made in the same way as in 
the simplex method, e.g., by the "ring around the rosy" method. The new 
tableau is given in Table 8-6. 

In the new tableau only хв: < 0. Therefore аз is removed at the next 
Step. The vector to go in is determined from 


(1/3. E 3j 
ES yp 


Thus a4 goes in. The new tableau is given in Table 8-7. 
All za; are now >0, and the optimality criterion is satisfied. Hence an 
optimal solution has been found. ‘The graphic solution of the problem is a 
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TABLE 8-7 — TABLEAU 3 


ĉj —3 —1 0 0 
Vectors 
св іп b al a2 аз a4 
Basis 
0 as 1 1 0 —3 1 
7-1 a2 1 1 1 —1 0 
2j — 6j _1 2 0 1 0 


simple matter and we can easily see that this answer is correct. Although 
we were using criteria which applied to the dual problem, the actual com- 
putations were carried out, completely in terms of the primal. Note that 
the value of the function to be maximized did not increase at each iteration. 
Instead, z decreased at each step. There is no reason why it should have 
increased since the basic solutions were not feasible until an optimal solu- 
tion was reached. In fact, z should decrease or remain unchanged at each 
iteration since when using the dual simplex method, we always have 
z = Z, and Z is being minimized in the dual problem. 


8-11 A geometric interpretation. An interesting geometric interpreta- 
tion can be given to the dual linear programming problem and the dual 
simplex algorithm. If we write the primal as 


n 


y» ajay = b, 


j=l 
x 2 0, (8-89) 
maxz — €x, 
then the dual problem is 
waj > cj, jm و ر‎ (8-90) 
minZ — b'w. 


The activity vectors a; in the primal become normals to half-spaces 
representing the constraints in the dual. The requirements vector b of 
the primal becomes a normal to the hyperplane Z — b'w in the dual. 
The actual geometry is best illustrated by means of an example. 
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Ficure 8-5 


Ехлмрік: Given the linear programming problem 
giare xgag = b, £z; 2 0, A 58/ dio хәб, 
maxz = —3x; — 473 — 223, 
where 
bs[—12, |a: (2,—2, 83 = [-2, 1, аз = [+2, 1), 
a, = [1,0, ^ as — [0,1]. 


‘The dual problem is: 
20, — 2v, > —3, 
—2u — ш> —4, 
—20; + w, > —2, 
wi 2 0, 
ш> 0, 


min Z = —ару + 2ш». 


Because of the appearance of slack variables in the primal, the dual 
variables are non-negative. 

‘The requirements-space configuration for the primal is shown in Fig. 8-5. 
For the dual problem, the convex set of feasible solutions in wy102-5pace is 
represenved by the shaded convex polyhedron shown in Fig. 8-6.* When- 
ever two of the dual constraints hold as strict equalities, the vectors normal 
to the constraints can be imagined to form a basis for the primal if fhe 
normals are linearly independent. In w,w-space, the point w where two 


* When illustrating graphically a vector normal to a line, we shall find it 
sometimes convenient to have the vector originate on the line rather than at 
the origin. This is done in Fig. 8-6 for greater clarity, 
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i 


Figure 8-6 


dual constraints hold as strict equalities is the intersection of the two lines 
representing these two constraints. A basic solution to the primal problem 
ean then be associated with the intersection of each pair of bounding lines 
for the half-spaces representing the dual constraints. 

_ There can be 5!/3!2! = 10 basic solutions to the primal, All these exist 
and are represented by the numbered points in Fig. 8-6. For example, 
point 1 corresponds to having а, аз in the primal basis. The only feasible 


` basic solutions to the primal are those corresponding to points 3, 6, 10, 


The points corresponding to feasible solutions to the dual and to basic 
solutions to the primal are the extreme points 1, 2, 3, 4, 5 of the convex 
polyhedron. Only one of these points, 3, also corresponds to a feasible 
solution to the primal. This then should be an optimal extreme point for 
the dual. On considering the dual objective function Z = b'w, we see that 
this is indeed the case. Using the dual simplex method, we move from one 
extreme point of the convex polyhedron to an adjacent one until an optimal 
extreme point is reached. At this point, the corresponding solution to the 


primal becomes feasible and, hence, optimal. 


8-12 A primal-dual algorithm. When it is necessary to add artificial 
variables to a linear programming problem, both the two-phase method 
and Charnes’ —M-method first drive the artificial variables to zero and 
thus obtain a feasible solution to the problem before working towards 
optimality. Unfortunately, in the process of driving the artificial variables 
to zero, they are not necessarily working towards optimality because the 
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criterion used to select the vectors to enter the basis is only concerned with 
driving the artificial variables to zero. Hence at the end of Phase I (and, 
similarly, when the artificial variables are zero in Charnes’ — M-method), 
the resulting feasible solution may not be anywhere near optimal. Conse- 
quently, a large number of iterations may be required in Phase II to 
achieve optimality. 

It would be very desirable if both objectives could be attained in Phase 
I, so that it might become possible to reduce considerably the total number 
of iterations. A great deal of work has been expended in trying to develop 
such an algorithm. The primal-dual algorithm to be discussed in this 
section is a result of such efforts. 

We have seen that the dual simplex algorithm can be used in certain 
circumstances to eliminate the necessity of introducing artificial variables. 
This algorithm does not have general applicability because it is often not 
easy to find a basic solution with all 2; — c; > 0. The primal-dual 
algorithm developed by Dantzig, Ford, and Fulkerson [3] does introduce 
artificial variables into the primal and hence does require a Phase I. The 
vectors to enter the basis in Phase I are chosen in a way quite different 
from that employed previously. The dual problem is used to determine 
which ‘vectors can enter the primal basis. The computational procedure 
is such that when Phase I terminates we have found an optimal as well as a 
feasible solution to the given problem. 

The procedure is rather interesting. We begin with a solution to the dual. 
This solution releases certain legitimate vectors of the primal for insertion 
into the primal basis. Then we work on the primal until the optimality 
criterion is satisfied for the vectors released to enter the primal basis. A 
new solution to the dual can be found which releases one or more additional 
primal vectors for entry into the basis. Each time a new solution to the 
dual is found there is a strict decrease in the dual objective function. 
Complementary slackness is maintained at each step so that when a 
feasible solution to the primal is found it is also optimal. Hence, in the 
primal, we are working on feasibility and optimality at the same time. 

We shall assume that the primal problem is 


Ax = b, XU maxz — cx. (8-91) 


It has been seen in the discussion of the dual simplex algorithm that it 
may not be easy to find an immediate solution to the dual of (8-91). 
Beale [1], however, has suggested a very simple modification of (8-91) 
which leads to an immediate solution for the dual. Suppose that we 
replace (8-91) by a modified primal problem, 


[, | l2] = NI [ro х] > 0, maxz = cx. (8-92) 
0 AJix b 


€. 
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We have introduced an additional constraint zo + Xj j= bo and an 
additional variable zo > 0 which appears only in the new constraint. 
The price of хо is zero. It is assuined that bo can be made arbitrarily large 
so that the new constraint places no additional restrictions on the 2;, 
f= 1,.0.,m For hand computations it is unnecessary to give a numerical 
value to bo. 

The dual of (8-92), called the modified dual, is 


ЕНИ 
VY A'Jiw c’ (8-93) 
min Z = bowo + b'w; 


the components of w are unrestricted in sign, but the first constraint of 
(8-93) requires wo > 0. However, we can immediately obtain a solution 
to this dual problem. One such solution is 


w=0; шо = MAX Cj, 7=0,1,...,1; 


where 
co = 0. (8-94) 


After adding surplus variables to the modified dual [Eq. (8-93)], we can 


write ; 
1 o][w] |t 9 [zl hi M (8-95) 
1 А о] ce 

and [w,o, Ws] > 0. If wo = 9, any solution to the modified dual problem 

(8-93) will also be a solution to the dual of (8-91). 


In Section 8-8, we noted that if x'w, = 0, or equivalently for dual prob- 
lems [Eqs. (8-92), (8-95)], if 


торо + X'W = 9, (8-96) 


then Z = z, assuming that [wso, We] 2 0 and x is any solution (not neces- 
sarily feasible) to the primal. If [zo, x] is a feasible solution to the modified 
primal and (8-96) holds, then [xo, x] is an optimal solution to the modified 
primal, and [wo, w] is an optimal solution to the modified dual. If, in 
addition, wo. — 0, then z — €x — Z — b'w, and from (8-96), x'w, = 0. 
Consequently, x is an optimal solution to the given primal problem, and 
ts dual. If we have a solution to the modified 


w is an optimal solution to i \ о 
ble solution to the modified primal such that 


dual with wo z 0 and a feasi 
(8-90) is satisfied (this implies то = 0), then 


2z =Z = bow + b'w. (8-97) 


| 
t 
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and in every case, 00 > 0. Now 


Z = (ibo, W')[bo, b] 
= (wo, W’)[bo, b] + balbo, b] (8-104) 
= Z+ 62", 


the primal has no solution. This can also be seen directly from the primal: 
Tf all 2 — c¥ > 0 and 2* < 0, we know that there is no feasible solution 
to the primal. 

Assuming that 00 ғ co, we obtain a new solution to the dual; this can be 
characterized by the new surplus variables ts; = wy; + boler — cf. 
There will be at least one ûs; = 0 for which Ws; was not in 5 because of 
the manner in which бу was Chosen. Hence, at least. one new vector af? 
which was not in the previous set P will be released to enter the primal 
basis. Consider the new set P of vectors a!” for which ûe; = 0. This new 
set will include at least one a” not in P and also those vectors in P for 
which z* — сў = 0 (in particular, this means that P includes the vectors 
from P in B). 


into the primal basis, etc. It is important to note that after à new solution 
to the dual is obtained, the initial tableau for the new restricted primal is 
the optimal tableau for the previous restrieted primal. 

We have noted previously that when z* — » we have an optimal 
solution to the primal, and maxz = Z for the dual, provided wo = wo = 0, 
If wo z 0, the primal has an unbounded solution, If at any stage 2* < 0, 
гў — Gi 2 0 for all j, the primal has no feasible solution. This covers 


1 Interestingly 
enough, the whole Primal-dual algorithm can be performed by means of 


the Simplex tableau. One only has to add an extra row which has Z = z 
in the column headed bP and sj in the ај? сопе, The pertinent new 
equations are (8-103) and tes = wy; + 0(2 — cf). 
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At the present time, not enough work has actually been carried out to 
show definitively that the primal-dual algorithm is really superior to the 
two-phase method, although it would seem so theoretically.* 


8-13 The primal-dual algorithm; an example. Let us illustrate the use 
of the primal-dual algorithm by solving the problem 


zı + 222 2, 
2 + 312 < 3, 
tı t2 > 0, 
maxz = тү + 622. 


After adding slack and surplus variables and annexing the additional 
constraint needed to find an initial solution to the dual, we obtain the 
modified primal 
Zo + zı + 22 + ta +24 = bo, 
zı + 22 — %3 =2, 
a + 8ш2 +24 = 3, 
x; > 0, JE 0-1,4, 


тах г = ту + бхә. 


For the initial solution to the dual, wo = max cj, c; > 0; hence, 
wo = max (1,6) = 6. Thus w = 6 and W, = wol' — €' or w = 
6—1 = 5, w = 6 — 6 = 0, шз = 6 — 0 = 6, ш = 6 — 0 = 6. 
Also, Z = wobo + wb = wobo = 6bo. The initial tableau is shown in 
Table 8-8. The last row gives the surplus variables for the dual problem 
and, in the first position, the value of Z. We begin with all artificial 
variables in the primal basis. Only ws2 = 0; hence, in the first restricted 
primal problem, only a? can enter the basis. In this first tableau, it is 
convenient to list the actual prices of the variables in the first row as an aid 
in obtaining the initial w,;-values. The prices actually used are, of course, 
сў = 0 for any legitimate vector and cf = —1 for any artificial vector. 

The vector to be removed from the basis is determined in the usual way. 
It is assumed that bo is so large that (0 will never be removed if it is possible 
to remove any other vector. From Table 8-8 it is clear that 95 should be 


* A recent master's thesis by Richard Mills (June 1960) at the School of 
Industrial Management, MIT, was devoted to a study of the efficiency of the 
primal-dual algorithm, A code to solve linear programming problems on the 
IBM 704 by the primal-dual algorithm was developed. Several problems were 
solved by using this code and the RAND code for the revised simplex method. 
The results, however, were inconclusive. In one or two cases, the primal-dual 
algorithm required a little less time than the revised simplex method, while in 
other cases it was somewhat more time-consuming. 
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TABLE 8-8 — TABLEAU 1 
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e; 0 1 6 0 0 
d, heri in p? a af af? af? af 
—1 qi? bo pm 1 1 1 
ay qi? 2 0 1 lol 0 
ip 90 3 0 1 3 0 1 
2-—d —5 — bo —1 —3 тё о —2 
Wsj 6bo 6 5 0 6 6 
TABLE 8-9 — TABLEAU 2 
8 * Vectors in а› (1) a) а) а) di 
ch, Basis b аб al аз аз 4 
zT 9 bo 1 1 4 0 1 $ 
—1 ai? 1 0 3 0 -—1.| -4 
0 p 1 0 PUER 1 0 EN 
zt — с —bo =] —4 0 0 —$ 
ш; bo 6 5 0 6 6 
[n 2450 21 0 0 6 43 


removed. “After inserting a’ into the basis, we obtain Table 8-9. Now 
the 2 — c > 0 for the vectors allowed to enter the primal basis, 
for aj). Hence, we must find new values for the %,; For those z 


€. < 0, we compute w,;/(cf — 


ratio 0. Only 20 — cj, zt — сї, zi — ci are <0, and 
бо уз ый б. Wa 15, Wea 
Е Cha EF ШИ Нлл АСТ тая 


2) 7), and pick the smallest ratio. Call "his 
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TABLE 8-10 — TABLEAU 3 


ch, bue е p? ay? ai 
ШЕ a Bo tae 1 1 
0 RU 3 0 L 
0 ay) 3 0 i 
zt — сў —bo +2 | —1 1 

Wej 24bo | 21 44 

Ф, 4} 0| 0 0 nj 24 

TABLE 8-11 — TABLEAU 4 
ch, Mer in pi? аб a? | a? a ii | 

0 ay? bo — 2 1 0 0 2 1 

0 al? à 0 1 0 —$ | — 

0 ay? i 0 0 1 i i 
a oof 0 0 0 0 0 0 

Wij 4} 0 0 0 1} 24 


hence 0 = 38. Then @,; = waj + 0 — of); Z = Z + 0e*. For ex- 
ample, 2 = 6bo + 42(—bo) = 2400; Ф = 6 + 48(—4) = 43. The tj 
are also given in Table 8-9. Only û,ı, ûsa = 0. Consequently, at the 
next step, only a(? can enter the basis. The remaining tableaux are shown 
in Tables 8-10 and 8-11. At the last step, al enters the basis, and we 
Obtain a feasible and optimal solution to the modified primal. Since 
wo = wo = 0, we also have an optimal solution to the given primal 
problem. It is тү = 2, тә = $, г = Z = 4}. The correctness of the 
result can easily be checked by solving the given problem graphically. 
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For this particular example, the primal-dual algorithm certainly did not 
reduce the amount of work required to solve the problem. However, one 
should not judge the merits of the method on such a small problem, which 
was introduced only for the purpose of illustrating the way in which the 
primal-dual algorithm can be used. 
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PROBLEMS 


8-1. Give the dual of the following problem in a form such that the dual 
variables are non-negative. 
22, + 312 + 5з > 2, 
3214 22+ 73 < 3, 
2 + 4x9 + 623 < 5, 
11, 12.13 > 0, 
minz = 271 + 222 + 423. 


8-2, Give the dual of the following problem in a form such that the dual 
Variables are non-negative. 
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321 + 723 + 823+ 5z4 + 25 = 2, 

2214 z2 + 3x3 + 214 + 915 = 6, 
21, £2, 23,24 > 0, z5 unrestricted, 

maxz = 62; + 412 + z3 + 724 + 525. 

8-3. Give the dual of the following problem in a form such that the dual 
variables are non-negative, and no strict equalities appear in the dual constraints. 
2.411 + 3.212 + 423 + 7.2%4 = 21, 

Зх + 1722 + 8023 + 224 < 48, 
zı a > 0, z3, z4 unrestricted, 
maxz = 1.821 + 2.422 + 6x3 + ха. 
8-4, Using the duality theory, solve the following linear rogramming prob- 
lem, and verify the result by solving the problem graphically. 
1821 + 1622 > 0.5, 
O.1a1 + 0.222 < 4, 
zı + 30:9 < 50, 
l4zi + тә > 0.1, 
zı + 0.0522 < 6, 
21,22 > 0, 
maxz = 3х1 + 222. 
8-5. Using the duality theory, solve the following linear programming prob- 


lem: 
311 + 512 + 423 > 7, 


бху + z2 + 3x3 > 4, 
Тху — 2x2 — 13 < 10, 
zı — 212 513 2 8, 
4x1 + 729 — 2x3 2 2, 

21, 22, 23 2 0, 
minz = 3z; — 252 + 413. 


8-6. By means of the duality theory, solve and illustrate geometrically the 
following linear programming problem: 
zi > d 
zid- ж € 7, 
zı + 212 > 10, 
z2 < 3, 
zjz2 2 0, 
maxz = Зх + 212. 
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8-7. Starting with the primal problem in the form of (8-13), and using argu- 
ments of the type developed in Section 8-3, show that if the primal problem has 
an optimal solution, so does the dual. Construct the solution to the dual. Work 
with (8-13) directly, i.e., do not convert it to the form of (8-7). 

8-8. Prove that a necessary and sufficient condition for the primal and dual 
to have optimal solutions is that both have feasible solutions. 

8-9. For the dual problems (8-33), (8-34); (8-35), (8-36); (8-37), (8-38); 
(8-39), (8-40), show that if B is an optimal basis matrix for the primal and cg 
contains the prices in the basis, then cgB—! is an optimal solution to the dual. 
Note that, in general, the dual variables will not be non-negative here. 

8-10. It was shown in the text that the dual of Ax = b, x > 0, maxz = cx, 
has unrestricted variables. However, if some slack and/or surplus vectors ap- 
pear in A, show that the dual variable for a constraint having a slack variable 
is non-negative, and that the dual variable for a constraint having a surplus vari- 
able is nonpositive. Hence, demonstrate that the only dual variables which are 
really unrestricted are those corresponding to constraints which were originally 
equations. In this way, show that the dual of any linear programming problem 
is unique (except for trivial changes in form), and is independent of the particular 
manner in which we write the primal. 

8-11. If in an optimal solution to the primal the kth constraint is an equality, 
under what conditions is it true that the kth constraint of the dual is a strict 
inequality? Assume the constraints are numbered as in (8-46). When does the 
above not hold? How are these results related to alternative optima in the primal 
and to degencracy? 

8-12. Consider an optimal basic solution to the primal and to the dual of the 
type constructed in Section 8-8. Show that if an optimal basic solution to the 
primal is degenerate, then there are alternative optima in the dual. Furthermore, 
if there are alternative optima in the primal due to some z; — c; = 0 for aj not 
in the basis, then the corresponding optimal solution to the dual will be degen- 
erate. 

8-13. Prove that if the primal has an optimal solution, then there is an 
optimal solution x to the primal (not necessarily basic) and an optimal solution 
w to the dual such that if 2; = 0, then w > 0, and if тє = 0, then w; > 0 
Hint: After a review of Section 8-5, use the results of Problem 8-12 to prove 
this. Consider first the case where the primal has a unique optimal solution 
which is not degenerate. Then consider the case where the primal has alternative 
optima, but none is degenerate. Finally, consider the case where degeneracy 
appears in an optimal solution to the primal and where, in addition, there are 
alternative optima. 

8-14. Construct an example different from that given in Section 8-6 to show 
that both the primal and dual problems may have no solution. 

8-15. Consider the primal problem Ax = b, x > 0, maxz = cx. Suppose 
that we multiply the kth constraint by the scalar А > 0. If w is an optimal 
solution to the dual of the above primal problem, what is an optimal solution to 
the dual of the primal problem whose kth constraint is multiplied by A? Hint: 
Note that the optimal solution to the new primal is unchanged, 
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8-16. Assume that we are given the primal problem Ax = b, x > 0, max 
z = cx, and an optimal solution, w, to the dual of the problem. Suppose that 
we form a new primal problem by adding (А # 0) times constraint k to con- 
straint r. What is an optimal solution to the dual of this new problem? 

8-17. Consider the primal of Problem 8-16. Suppose that we add a multiple 
^ » Oof row k of A to с. How does an optimal solution to the resulting primal 
compare with an optimal solution to the original problem? Given an optimal 
solution, w, to the dual of the original problem, what is an optimal solution to 
the dual of the new problem? 

8-18. Solve the following linear programming problem by the dual simplex 
algorithm: 

231-22 > 4, 
zı + 7222 7, 
zı, z2 > 0, 


minz = 2 + 22. 


8-19. Solve the following linear programming problem by the dual simplex 
algorithm: 
2x1 412 + 523+ та > 10, 
321 — 22-713 — 214 2 2, 
5x1 + 222 + хз + 04 > 15, 
ту, 12, 23, 14 > 0, 
minz = 301 + 2z2 + za + 424, 


8-90. Solve Problem 8-19 by the simplex method, using the two-phase tech- 
nique, and compare the number of iterations required in both methods. 

8-21. Solve the following linear programming problem by the dual simplex 
algorithm: 
4х1 — x2-- 2z3 + 3914 — 5z5-L 676 > 2, 


zı + Зло — 4x3 + 5r4 — 25+ 26 2 18, 

Зал + 722 + хз — 214 + 25 — 926 2 7; 

блі — 3x2 + 8тз — 414 625+ zo 2 6, 
11,22, 23, 4 2 0, 

minz = zı + 322 + 723+ 5x4 + 2х5 + 816. 


8-22. Solve Problem 8-21 by the simplex method, using the two-phase tech- 
nique. Compare the number of iterations required in each case. ( 

8-93. Lemke has suggested the following procedure as а possible technique 
for getting started on the dual simplex algorithm: We are given the problem 
Ax = b, x > 0, maxz = cx. Find m linearly independent activity vectors 
from A. Let the vector b* be a linear combination of these vectors, with each 
scalar coefficient positive. Then solve the problem Ax = b*, x > 0, maxz = 
cx by the simplex method. In the optimal solution replace b* by b. This 
gives a basic (although not necessarily feasible) solution to the original problem, 
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with all z; — с; > 0. The dual simplex algorithm can now be applied. Discuss 
why this procedure works. Are there any cases where difficulties may arise? 
Can this be considered a practical procedure for getting started on the dual 
simplex algorithm? 

8-24. In the special case when all су € 0, Dantzig has suggested a more 
Straightforward way of getting started on the dual simplex algorithm. His 
method involves the direct use of the dual. Begin with the dual basis B = —I, 
so that w,; = —c; > 0. Since a!,..., a" should be in the dual basis, use an 
arbitrary rule to insert these vectors. Discuss in detail how this method yields 
an initial solution for the application of the dual simplex method to the primal. 
Also, suggest a rule for inserting the a‘. Does this method appear practical? 

8-25. Using Charnes’ perturbation method, resolve the degeneracy problem 
in the dual simplex algorithm. Hint: Applying Charnes’ method directly to the 
dual problem and using only a set of linearly independent vectors, we should 
replace c' by c' — У de; or cj(e) = cj — d. Thenzj(e — cole) = cs(6y; — 
cj-F €. What is the rule for choosing the vector to enter? Note that, in this 
case, one must be concerned about the question whether the lowest powers of € 
are in the basis. If, for example, сү is not in the basis, e! appears only in zi(e€) — 
с1(е). Show that a tie will never be resolved, on the power of € corresponding to 
the vector —e; which is to leave the basis, and consequently, we need not be 
concerned about losing from the basis the power of e for which the tie was re- 
solved. 

8-26. For each of the pairs of dual problems in Section 8-4, show that the 
dual of the dual is the primal. 

8-27. 1f the addition of only a relatively small number of artificial variables 
to a given problem permits us to use the two-phase method, would this method 
or the primal-dual algorithm scem to be more efficient? Why? Under what 
circumstances can one expect the primal-dual algorithm to be better than the 
two-phase method, and conversely? Note that all these alternatives are es- 
sentially conjectures, and that no real evidence has been accumulated which 
allows any definite, conclusions. 

8-28. Using the primal-dual algorithm of Section 8-12, solve and illustrate 
graphically the following problem: 


311 + 222 > 6, 

^ 21+ 622 > 3, 

21,12 > 0, 
maxz = zı + 209. 


8-29. Using the primal-dual algorithm, solve the following problem: 


8x1 + 5x2 — 6rs + 2r4 + 405 = 27, 
zi + 212 + 323 — 724 + 615 22, 
91i — 4z2 + 2r3 + 5r4 — 2r5 = 16, 
т;> 0, мен Ду. 15, 
тах г = 72; + 212 -+ тз + 4z4 + 6zs. 
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8-30. Solve the following problem by means of the primal-dual algorithm: 


x1 + 5тә -+ 2z3 + Tra + 415 + 0x6. — 25, 

Зір — 212 — za + 815 1-16 = 9, 

4x) х2 + 723 — 314 — 215 2 3, 

2x1 + 612 — 3x3 + 425 + 26 > 5, 
22020, j=1,...,6, 


maxz = 32, — 2х2 | 4x3 + та + 6x5 + 926. 


8-31. Given an optimal tableau for the linear programming problem Ax = b, 
x > 0, maxz = cx, solved by the simplex method, show how to find an optimal 
solution to the dual from this tableau. 

Consider both cases: (a) where it is necessary to add artificial variables, and 
(b) where artificial variables are not needed, 

8-32. Consider the primal-dual algorithm discussed in Section 8-12. How do 
we know that on adding the constraint zo + Ej zj = bo, we cannot have 
m + 1 vectors from А in an optimal solution? Hint: Recall that bo can be arbi- 
trarily large. If zo is not in basis, the other zs; must depend on bo. As bo is 
increased, z will either increase or decrease. 

8-32. Why must we be concerned about degeneracy in the primal-dual 
algorithm, when it was shown that, for the dual, Z strictly decreases at each 
dual iteration? Hint: Is the value of Z changed for each primal iteration? 

8-33. Given an optimal basic solution to a linear programming problem. 
Can one obtain an unbounded solution by changing the requirements vector? 
What is the geometrical interpretation of this result? Hint: Is there a solution 
to the dual? 

8-34. Prove that if a given basic feasible solution to some linear programming 
problem is optimal, the same basis vectors will yield an optimal solution for any 
requirements vector which lies in the cone spanned by these basis vectors. 

8-35. Show that an optimal basic solution to Ax = b, x 2 0, maxz = cx 
is also an optimal solution to the same problem if it is converted to the form 
Dx, < d, x, > 0, maxz = cx, before application of the simplex method. 
Note that if there are any equations in the original formulation of the problem, 
D will have more rows than A; this implies that there will be more basic vari- 
ables when the second formulation is used. What will these additional basic 
variables be? Hint: What is the rank of D? 

8-36. After a new solution to the dual has been obtained, under what cir- 
cumstances will more than a single additional vector be released to enter the 
restricted primal basis in the primal-dual algorithm? If more than a single 
vector is released to enter the restricted primal basis, is it expected that this will 
increase or reduce the total amount of time required to solve the problem? 

8-37. Solve the example of Section 8-13 by the primal-dual algorithm without 
annexing the constraint zo + oz; = bo, that is, find a solution to the dual of 
the given problem without using wo. 

8-38. Why did we include the artificial vector 9 in the example of Section 
8-13 when aj" is a unit vector? 
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8-39. Discuss the simplifications introduced into the primal-dual algorithm 
when one can find an initial solution to the dual without annexing the addi- 
tional constraint to the primal. 

8-40. It has been shown that if xs = B-!b, z = csxg is an optimal basic 
solution to some linear programming problem, then ж’ = cgB 7" is an optimal 
solution to the dual. Will this optimal solution to the dual always be a basic 
solution? 

8-41. We have shown above that a given optimal basic feasible solution to 
the linear programming problem Ax = b, x > 0, maxz = cx uniquely de- 
termines an optimal solution to the dual if r(A) — m. Show that if there is 
redundaney in the constraints, then an optimal basie solution to the primal 
with one or more artificial variables in the basis does not really uniquely deter- 
mine the dual variables in the corresponding optimal solution to the dual 
caB7!, that is, there is an infinite number of solutions to the dual, which yield 
the proper (z; — cj)-values for the legitimate primal vectors. 


CHAPTER 9 
TRANSPORTATION PROBLEMS 


"Under a world of whistles, wires and steam 
Caboose—like they go ruminating through 
Ohio, Indiana—blind baggage— 

To Cheyenne tagging . . . maybe "Kalamazoo." 
Hart Crane. 


9-1 Introduction. A certain class of linear programming problems, 
known as transportation type problems, arises very frequently in practical 
applications. In Chapter 1, the general transportation problem was 
formulated as follows: A product is available in known quantities at each 
of m origins. It is required that given quantities of the product be shipped 
to each of n destinations. The minimum cost of shipping a unit of the 
product from any origin to any destination is known. We wish to deter- 
mine the shipping schedule which minimizes the total cost of shipment. 

Let a; be the quantity of the product available at origin 7, and b; the 
quantity of the product required at destination j. The cost of shipping one 
unit from origin i to destination j will be written cij. We shall assume that 


Же (9-1) 


iml j=l 


so that the total quantity required at the destinations is precisely the same 
as the amount available at the origins. Then if хуу is the quantity shipped 
from origin i to destination j, we wish to find zi; 2 0 which satisfy the 


m + n constraints 


D ty = an a; > 0, i= l,m, (9-2) 
j=l 
DEI E (9-3) 
i=l 


and which minimize 
z= 20 Cijtij. Sp 
id 
From Eq. (9-1) it follows that, in our present formulation of the problem, 
the strict equality sign holds in every constraint. The formulation in 
273 
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Chapter 1 allowed the constraints (9-2) to have a < sign since the total 
amount available at the origins was permitted to be greater than the total 
amount required at the destinations. In reality, however, the formulation 
in Chapter 1 was no more general than our present one. Since any trans- 
portation problem can be converted to the form of Eqs. (9-2) through (9-4) 
(see Section 9-12), there is no loss in generality if we concentrate our at- 
tention on problems which can be cast into this form. 

First, let us convert the constraints of the transportation problem into 
our standard matrix form for a linear programming problem, Ax = b. 


Write 


х= [ту,..., Lin, ai,» +s Dan ss) Ton]; 1 
(9-5) 
b = [a1,..., Gm, 01, . . . , bs]. i 
If the constraints are written as 
zirt zı2 omn = а, 
T21 F eee H Zan, = ag 
Tmt PUET E Tain = Om 
zuo | Tznd-E al = b, 
Lin + Z2n + ese F Za. = bn, (9-6) 
they take the form Ax = b provided 
1, 0 0...0 
0:1: 1, 40 $50 
ip et опао таго m rows, (9-7) 


0. 0 301. 
I, IL, I,---1,{} rows 
ua Bm 

mn columns 


where A is an (m + n) X (mn) matrix. Of course, 1 is the sum vector and 
the subscript n implies that it has n components. Note that a given 
variable appears in two and only two of the constraints. 

Equation (9-7) can be used to provide a rigorous definition of what we 
mean by a transportation problem. Any linear programming problem 
Ax = b,x > 0, maxor minz = cx will be called a transportation problem 
if it can be cast into such a form that the matrix of the coefficients A has 
the structure (9-7). 
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ExawPnE: For the specific case of two origins and four destinations, 
Ax — b becomes 


Zu 
INE 1d) 0070-0 а 
112 
0.0005 UP тка а2 
13 
1050000110000 | | 
01000 1 0 Offa bs 
0010 0 0 1 O0||2 bs 
000 1 0 0 о 1[| 2з ba 
T24 


We have cast the transportation problem into the form of the standard 
linear programming problem, and hence the simplex method could. be 
applied directly to any transportation problem. However, because A has 
a very simple and special structure, it is possible to develop algorithms for 
solving transportation problems which are much more efficient computa- 
tional procedures than the simplex method. This chapter will be devoted 
to a study of some of these special algorithms. 


9-2 Properties of the matrix A. The matrix А of (9-7) has m -+ n rows 
and mn columns. Column (i — 1)n + j of A will be-denoted by pij. We 
shall find it convenient to use a double-subscript notation for the activity 
vectors just as we are using a double subscript on the variables. Each 
pi; is an (m + n)-component vector, and only two components of pij 
differ from zero. In fact, we can write 


ру = eic ena (9-8) 
where the e; are the unit vectors for Bats; 

The rows of A fall naturally into two sets: the first m rows which come 
from the origin (or source) constraints and the last n rows which come 
from the destination constraints. We shall call these two sets the origin 
(or source) rows and destination rows of A, respectively. From (9-7) note 
that adding the first m rows of A (the origin rows) yields lemn), and adding 
the last n rows of A (the destination rows) also yields 155). Thus the sum 
of the first m rows of A minus the sum of the last n rows of A gives a null 
vector. Consequently, the rank of A is less than m + n. In fact, the rank 
of A is m+n — 1, as we can easily show by finding a determinant of 
order m + n — 1 in A which does not vanish. Consider the matrix D 
formed from A by taking columns n, 2n, ..., mn, l,...,n — 1 and 
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rows ],..., m +n — 1 (the last row of A being omitted). Then 


Ө КО ДН ДЕД Турагы: Ва iva (9-9) 
OT 5T; y 0 
Since D is a square matrix of order m + n — l,r(A) = m-- n — 1. 
Let us denote the origin rows of A by s‘, i = 1,..., m, and the destina- 
tion rows by @?, j = 1,..., n. Then the relation expressing the linear 


dependence between the rows of A can be written 


b» s! — У d = 0, (9-10) 


iml j=1 


It is important to note that the coefficient of every row vector is either 
+1 or:—1. This means that any row of А can be expressed as a linear 
combination of the remaining m -+ n — 1 rows. In other words, we can 
cross out any one row of A, and the resulting (m + n — 1) X (mn) 
matrix will have rank m + n — 1. This follows because we know that at 
least one set of m + n — 1 rows is linearly independent, and by (9-10), 
the remaining row can replace any row of the linearly independent set, 
and the new set will also be linearly independent. 

What we have just proved is equivalent to saying that only m -- n — 1 
of the constraints (0-2), (9-3) are independent. Furthermore, we can 
remove any one constraint, and the remaining m + n — 1 constraints 
will be independent. Intuitively, all this is obvious. Since we are assuming 
that (9-1) holds, any set of туу which satisfies all but one of the constraints 
must automatically satisfy this remaining constraint. 

It is also interesting to note that the same sort of linear dependence 
holds for the rows of any (m + n) X k matrix R formed by choosing any 
k € mn columns of A. If we denote the source rows of R by s and the 
destination rows by 47, then (9-10) holds. In fact, (9-10) also holds for 
a single vector pij. As a particular case, assume that R is formed from any 
m+ n — 1 linearly independent columns of A. Then we know that r(R) = 
mt n — 1. Furthermore, we know from (9-10) that since any row of Risa 
linear combination of the remaining m + n — 1 rows, every sel of m +n — 1 
rows of R is linearly independent. This means that if any single row of R is 
crossed out, the remaining matrix of order m +- m — 1 will be nonsingular. 
Similarly, if we have an (m + n) X (m +n — 1) matriz R formed from 
т +n — 1 columns of A, and if there is one minor of order m+n — 1 in 
R which does not vanish (implying that r(R) = m +n — 1), then all m + n 
minors of order m + n — 1 in R are different from zero. 

` An interesting and important property of the matrix A is that every 
minor of A can only have the value +1 or 0. (This is sometimes referred 
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to as the unimodular property of the matrix A.) Let A, be a kth-order 
submatrix formed from any k different columns and Ё different rows of A. 
We wish to prove that 

lA = +1 or 0. (9-11) 


First note that each column of A; contains either two 1’s, a single 1, or 
no 1. If А, contains one or more columns of zeros, then clearly |A,| = 0. 
If, on the other hand, each column of A; contains two 1’s, then for every 
column in Ag, one of the 1’s must occur in an origin row, and the other in a 
destination row. Taking the sum of the origin rows minus the sum of the 
destination rows, we obtain a null vector; hence, the rows of A, are not 
linearly independent, and again |A;| = 0. Finally, if every column of A, 
contains one or two 1’s, and at least one column contains only a single 1, 
then we expand [A;| by any one of the columns which contain a single 1. 
This gives 1 

ЈА = А21, 


where |A;,_,| is a minor of A of order Е — 1. Now the same arguments can 
be applied to Aj ,. Either |А, | = 0 or [Aj 1| = -Е|А„—;|, etc. How- 
ever, we observe that every |A;| = 0, 1 since each matrix element of A has 
the value 0 or 1. Thus we have proved that every minor of A has the value 
+1 or 0. This very important property of the A matrix allows us to 
develop algorithms for solving the transportation problem which are much 
more efficient than the direct application of the simplex method. 


ExaMPLE: It is easily checked that the minors of the matrix A in the 
example on page 275 have only the values +1, 0. Consider the minor 
formed from columns 1, 2, 3 and rows 1, 3, 5. It is 


The minor formed from columns 2, 4, 6 and rows 3, 4, 5 is 
000 
1.0. 1| 2:0. 
00 0 
The minor formed from columns 1, 7 and rows 3, 5 is 
1 0 
0.1 


=1. 
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9-3 The simplex method and transportation problems. Let us examine 
the problems involved in applying the simplex method directly to a trans- 
portation problem. When the constraints are written in the form Ax — b, 
A does not contain an identity matrix. In fact, there are no unit vectors at 
all in A. Thus, to begin with a basis matrix which is an identity matrix, 
itis necessary to annex artificial vectors. The initial basis matrix will con- 
tain only artificial vectors. We noted above that there is redundancy in 
the constraints (9-2), (9-3); in addition, we saw that any one constraint 
can be dropped, and the remaining constraints will be independent. Al- 
though we could eliminate a constraint before applying the simplex 
method, we shall find it convenient to retain all m + n constraints. Then 
we add m + n artificial vectors. We know that an optimal solution to the 
problem must have one artificial vector in the basis at a zero level, i.e., 
every basie feasible solution to the set of constraints 


(A, 1) Š] =b (9-12) 


will contain at least one artifieial vector. The following important result 
immediately follows from the fact that r(A) — m 4- n — 1: An optimal 
solution to a transportation problem with m origins and n destinations never 
need have more than m +- n — 1 of the xij different from zero. 

It is not uncommon in practice to find transportation problems which 
have, for example, 25 origins and 1,000 destinations. For this problem, 
there are 1,025 constraints and about 25,000 variables. None of the 
presently available computer eodes for the simplex method will handle a 
problem of this magnitude. Even if they did, the solutions would require a 
great deal of time. It is quite clear that to solve praetical problems of this 
sort, a much more efficient algorithm than the direct application of the 
simplex method must be found. Interestingly enough, if we study theo- 
retically the application of the simplex method to the transportation 
problem, a number of simplifications become apparent, permitting the de- 
velopment of an algorithm which, with the aid of a large scale computer, 
ean easily solve a problem involving 25 origins and 1,000 destinations. 

Before going on, let us note that if (9-1) holds, the transportation 
problem always has a feasible solution and hence an optimal feasible solu- 
tion. Intuitively, this is obvious. There are many ways of obtaining a 
feasible solution. Clearly, 


= а NN (9-13) 


is a feasible solution. Another way is to begin with origin 1: We ship 
min (бу, ау) to destination 1. If b; < a;, we ship min (bs, a — 01) to 
destination 2, If b > a, we move to origin 2 and ship min (b; — ay, аз) 


9-3] THE SIMPLEX METHOD AND TRANSPORTATION PROBLEMS 279 


to destination 1, ete. Continuing in this manner, we finally satisfy all 
origin and destination requirements. A particularly interesting feature of 
this method is that, at each step, we satisfy either one of the origin require- 
ments or one of the destination requirements. Thus we obtain a feasible 
solution in which no more than m + n — 1 of the z;; are different from 
zero. 

We might also note that there is no feasible solution to (9-2) and (9-3) 
unless (9-1) holds. To see this we only need to sum (9-2) over ї and (9-3) 
overj. This requires that any solution to (9-2), (9-3) must satisfy 


ў» y Tij = 5 ai, (9-14) 


tet j=l i=l 
SS fas (9-15) 
j=l i=l j=1 


The same quantity appears on the left-hand side of (9-14), (9-15). Thus 
if there is a solution, (9-1) must hold. We have shown that a necessary 
and sufficient condition for a feasible solution to (9-2), (9-3) to exist is 
that (9-1) holds. 

Since a transportation problem has a feasible solution if (9-1) holds, it 
follows that the artificial variables can always be driven to zero in Phase I. 
Consider a basic feasible solution to (9-12) which is also a feasible solution 
to Ax = b. Let B be the basis matrix (of order m + п). In general, В 
will contain m + n — 1 of the руу and one artificial vector q which will be 
at а zero level in the basic feasible solution. Furthermore, any р;; in A 
can be expressed as a linear combination of the m + n — 1 vectors from 
Ain B. Thus, in reality, we never have to make explicit use of the artificial 
vector. For this reason, we shall refer to any set of m -- п — 1 linearly 
independent vectors from A as a set of basis vectors for the transportation 
problem. A particular vector from such a set of basis vectors will be 
denoted by pZ, and the corresponding basic variables will be denoted by 
28). 

Any vector р; from А can be written as a linear combination of the basis 
vectors in the following way: 


ру = Уур (9-16) 
af 


where 32,4 means a summation over the basis vectors. The two subscripts 
and superscripts on the yf are rather clumsy, but we use them for clarity. 
Fortunately, we do not need to write these yf very often; they are, of 
course, the numbers that would appear in the usual simplex tableau. We 
know that the artificial vector q is not needed in (9-16) so that y, = 0 
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for all 7, j, that is, a row of zeros in the simplex tableau corresponds to the 
artificial vector. 

We shall next prove the interesting result that every yf? is 0 or +1. 
Observe that (9-16) is a set of m + n linear equations in m +n — 1 
unknowns, the y?. In matrix form, it can be written Ry;; = pij, and R is 
a matrix formed from m + n — 1 linearly independent columns of A. 
Recall from Section 9-2 that any one row of R can be crossed out, and the 
resulting matrix will be nonsingular. Let us then drop equation 7 from 
(9-16). Note that the ith component of p;; contains a 1. The new set of 
equations obtained after eliminating the ith equation can be written 
Ту; = €j4m—1, Where еры is а unit vector containing m + n — 1 
components and Т is obtained from R by deleting row i. When a row of 
pij containing a 1 is crossed off, we are left with a unit vector, But T is 
nonsingular, so we can write 


у = Тер 1 = тул, (9-17) 


where турт is the (7 + m — 1)-column of T71. However, each com- 
ponent of 7;.,.,,-, is some minor of order m + n — 2 from T divided by 
|T|. Furthermore, |T| and the minors of T are minors of A. Consequently, 
each element of т; is +1, 0, and hence each yf? is +1, 0. 


9-4 Simplifications resulting from all yf? = +1or0. We are now ina 
position to obtain some important results. In going from one simplex 
tableau to the next, let us first consider the transformation formula for the 
basic variables. When the double subscript notation is used, it is 

af B 

Bag = ты — ab, ав رسک‎ ty, 2, 
Yst Yat 

where pst is the vector entering the basis, and р2, the vector to be removed. 
But by our above discussion, it must be true that ys = 1, yf = +1, 0 
so that 27, = 2%, or 22, + х8, Note that division is no longer needed. 
The new value of the variable is found from the original one by means of 
simple addition or subtraction. Suppose that the a;, b; in the requirements 
vector are integers. Then the basic variables.in the initial basic feasible 
solution to (9-12), x, = b, will be integers. Since the transformation for- 
mula for the basic variables requires only addition or subtraction, the basic 
variables in every basic feasible solution will have integral values. This 
means that the basic variables in any optimal basic solution will be in- 
tegers, i.e., we arrive at the following theorem. 


INTEGRALITY PROPERTY: If the aj, b; of (9-2), (9-3) are integers, then 
any optimal basic solution to the transportation problem will have the 
property that all positive x;; will be integers. 


| 
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This integrality property is peculiar to the transportation problem. 
The reader will recall that, we cannot, in general, expect that an optimal 
solution to an arbitrary linear programming problem will have integral 
values for the variables. In fact, if we require that the variables be integers, 
a linear programming problem usually becomes a nonlinear programming 
problem. Intuitively, this integrality property is expected to follow from 
the physical nature of the problem: If it is profitable to ship a fraction of a 
unit to any destination, it is profitable to ship as large a quantity as possible. 
Since an integral number of units is required at each destination, an integral 
number of units will be shipped. 

When numerical computations are performed by the simplex method, 
the bulk of the work consists of determining the ;; from the y;; at each 
iteration. For the transportation problem, the y7? are always 0, +1. This 
suggests that by not transforming the yf at each iteration, we might be 
able to avoid a large part of the work required in the simplex method. 
Fortunately, this is correct. It is instructive to examine more closely the 
problem of expressing any p; in terms of the basis vectors p¥,. If the 
p, for which yi? = 0 are omitted, our above discussion shows that the 


J 
relation expressing the linear dependence between p;; and the p, can be 


written ү 
pi = X(E)ps. (9-18) 


Each of the vectors руу, р2, has the form (9-8). Consequently, since the 
coefficient of each pZ, is +1, there must be one p, of the form pf, = 
е; + en u, whose coefficient is +1 in (9-18), so that there will bea 1 in the 
ith component of p;j.. Then if u = j, there must be a р? of the form 
PA, = е, елди, Û < m, whose coefficient is —1 in (9-18), so that the 
1 in the (m + u)th component of pë, will be cancelled out. Continuing in 
this way, we must reach, in a finite number of steps (Xm +n — 1), a 
vector р2,, of the form pz; = ew + emt) whose coefficient is positive in 
(9-18). The 1 in the wth component will cancel the wth component of the 
immediately preceding vector (whose coefficient is — 1), and the (m + j)th 
component of pë; will give the (m + j)th component of р;;. Thus we see 
that the expression of any pi; in terms of a set of basis vectors will have a 
remarkably simple form. Furthermore, since the representation of any 
vector in terms of a set of basis vectors is unique, the above representation 
is unique. 

The preceding discussion has shown that the representation of p;; in 
terms of the pP, can be written 


Pj = рї — piu + ри 2-5 ро, + pij: (9-19) 


Since in (9-19) we begin and end with a plus sign, and since the signs are 
alternately plus and minus, it follows that an odd number of basis vectors 
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at à nonzero level is always used to express any vector from A in terms of 
the basis vectors. 


Exampte: For the matrix A given in the example on page 275, the reader 
can quickly verify that the vectors Pii, Piz, P22, Pes, P24 are linearly 
independent, and hence any vector in A can be represented as a linear 
combination of these five vectors. Three vectors, P21, Pia, Pia, are not in 
the above set. We see that 


P21 = P22 — Pi2 + Pir, риз = Piz — P22 + pos, 
Pia = pis — Poo + paa. 


It is suggested that the reader write out the vectors explicitly to verify that 
all the intermediate 1’s do cancel. In each case, the expression of a vector 
in terms of the basis vectors has the form (9-19). Furthermore, an odd 
number of basis vectors appears at a nonzero level in every instance, but 
not every basis vector appears at a nonzero level in any one expression. 
However, each basis vector does appear in at least one of the three ex- 
pressions. 

Consider the following symbolic matrix of the Pi; (the basis vectors are 
circled). Note the interesting fact that each Pij not in the basis and the 
basis vectors which appear at a nonzero level in (9-19) form what we might 
call a loop in this matrix. 


ОЕР 
Poo sd j 
ње (а) (н) (Б) 


The idea introduced in the above example of representing the Pij asa 
symbolic matrix is very important. If we circle a set of m л =] 
linearly independent vectors їп the matrix ||pis||, we can see immediately 
how to express any other vector Pi; in this matrix in terms of the basis 
vectors. Beginning with руу, we move in row î until we encounter a basis 
vector рд, such that there is another basis vector in column « of the matrix. 
A coefficient of +1 is assigned to р2. Next, we move in column u from pë, 
to a basis vector pë, which has another basis vector in row v. A coefficient 
of —1 is assigned to pë,. Then, we find a basis vector РЁ which has another 
basis vector in column t, etc. Ultimately, we must close the loop with some 
basis vector p2;. In this way, we obtain (9-19) without having to com- 
pute explicitly the уе? 


— 5A کس س ج‎ RR 
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It is important to note that we did not imply in the above paragraph 
that any “path” formed in this way will ultimately close the loop. There 
may be some “blind alleys.” However, we are sure that there will be one 
path that will close the loop. Furthermore, there will be only one such path, 
i.e., the loop is unique. Observe that instead of finding at the first step a 
basis vector pF, in the same row as p;; such that there is another basis 
vector in column u, we can equally well find a basis vector pë; in the same 
column as p;j such that there is another basis vector in row s, and obtain 
precisely the same loop; however, we traverse it in the opposite direction. 


Ficure 9-1 


EXAMPLE: Consider a transportation problem having four origins and 
six destinations. It is easily shown that the nine vectors p11, P12; P32; P23, 
P14, P24, раз, рав, рав are linearly independent. One only needs to evaluate 
any 9th-order minor in the matrix formed from these vectors (this is really 
quite easy to do—try it). Thus, for this problem, any vector in the A 
matrix can be expressed as a linear combination of these basis vectors 
which are represented by dots in Fig. 9-1. Suppose that we wish to express 
рә in terms of the basis vectors. Consider Tig. 9-1, where the cell repre- 
senting рә is indicated by a circle. Beginning with pou, let us find a basis 
vector in column 1, p, which has another basis vector in row û. There is 
only one basis vector, ру, in column 1. In addition to P11, row 1 contains 
two other basis vectors, pis, P14. Note that there is a basis vector pa» in 
column 2 and a basis vector руу in column 4. If we choose рух as the next 
basis vector, we must follow the dotted path in Fig. 9-1. It is a blind 
alley and does not lead to a loop. On the other hand, if we choose p,4 as 
the second basis vector, we are led to the loop shown in solid lines. From 
this loop, we see that p2, can be written as a linear combination of the basis 
vectors in the following way: 


Por = Pit — Pig + P24- 


All other basis vectors have zero coefficients. 


Let us now turn our attention to the computation of the 2; — ¢;. For 
the transportation problem, it is convenient to use two subscripts here 
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also, so that we shall write Zij — бу. If cP, is the price corresponding to 
the basis vector p2, then 


Zij — су = D Vici — су (9-20) 
af 
or 
Zij — суу = L(+) — суу. (9-21) 


For the vector p;; of (9-19), we can write 
Zij — су = Ch — om d eA — ... moB, cw; — су. (9-22) 


If we arrange the cj; into а matrix llcill, then we can compute all the 
Zij — суу, using the same technique of finding a loop involving cij and the 
cH, that was used to express Pi; in terms of the р5,. For the above example, 
we see that 

#21 — Coy = C11 — C14 + C24 — со. 

In this section, we have examined the theory underlying the application 
of the simplex method to any transportation problem with the following 
results. Some remarkable simplifications are introduced into the trans- 
formation formulas by the fact that the yf are always either 0 or +1. If 
we arrange the p;; into a symbolic matrix |р], it is very easy to deter- 
mine the vf for a given basis without any computation whatever. Further- 
more, if the cj; are arranged into the same type of matrix, we can compute 
the zi; — ci; directly, using only the operations of addition and subtrac- 
tion. Here we have the key to the development of an efficient algorithm 
for solving transportation problems. 


9-5 The transportation-problem tableau. The results of the preceding 
section suggest that we should focus our attention on a matrix tableau 
involving m rows and n columns rather than on a simplex tableau with 
т п - 1 rows and mn + m-+-n--1 columns. Let us consider the 
tableau shown in Table 9-1. In cell (i, j) enter cij and Tjj. Note that if the 
Zij entered in the tableau represent a feasible solution, it must be true that 
addition of the тү; in row i yields aj, i = 1,..., т, Similarly, if we sum 
the z;; in column j, we must obtain bag = 1,..., m. Hence, all the con- 
straints are conveniently represented, and it is easy to check whether any 
set of тү; is a feasible solution by simply summing the rows and columns. 

Tn the last column we enter the origin availabilities and in the last row 
the destination requirements. It is also convenient to use D; as a heading 
for column j to indicate that this column pertains to destination j. Simi- 
larly O; is placed at the beginning of row i to indicate that this row pertains 
to origin i, We can, of course, also associate the vector руу with the cell 
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TABLE 9-1 — TABLEAU FOR TRANSPORTATION PROBLEM 


Di Da Dj D, a 


(i,j). However, we shall never make explicit use of these vectors, In the 
lower right-hand cell of Table 9-1, it is often desirable to enter the total 
amount to be shipped, i.e, Lai = Eb; 

When dealing with basic solutions, we know that no more than m -+ 
n — 1 of the туу in Table 9-1 will be positive, If more than m + — 1 
of the туу are positive, the vectors piy corresponding to these variables will 
be linearly dependent. Since we shall be concerned only with basic feasible 
solutions, no more than m + m — 1 of the z;y will ever be >0. Only the 
values of the basic variables will be entered in the tableau; i.e., we shall 
not fill in the zeros for the nonbasie variables. However, zero values of 
the basic variables will be written in. 

In the previous section, the notions of a path and a loop were introduced 
in an intuitive fashion. Now we shall find it desirable to make these con- 
cepts more precise. 

DIRECTED PATH JOINING TWO CELLS; A directed path from the cell (i, j) 

10 the cell (v, w) in Table 9-1 is defined to be an ordered set of cells | (i, j), 

Gh), (4, E), (4,7), - =, O, @)} or 10,3), (8,7), (s, D, ..., (0, w)} such 

thal any two adjacent cells in the ordered sets lie in the same row or same 

column, while any three adjacent cells do not lie in the same row or same 
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ү 
zu 


Ficure 9-2 


Figure 9-3 Figure 9-4 


column. Furthermore, each cell (except the last) must appear only once in 
the ordered sets. The cell (2, 7) is called the initial cell of the path, and 
(v, w) is called the terminal cell. 


It is convenient to be able to illustrate graphically a path connecting 
two cells in Table 9-1. To do this we simply join by line segments the 
ordered set of cells which form the path. The direction is indicated by an 
arrowhead on the line. These line segments will be called branches, 


DIRECTED BRANCH: A directed branch is a line segment joining an ordered 
pair of cells which lie either in the same row or the same column of Table 9-1. 
The first cell of the ordered pair is called the initial point of the branch, and 
the second cell is called the end point of the branch. 


A path, as we have defined it, may contain only a single element. If 
it contains more than a single element, it can be represented by a sequence 
of one or more branches. The end point of one branch is the initial point. 
for the immediately succeeding branch. In addition, each branch is orthog- 
onal to the branch immediately preceding it. A typical directed path from 
cell (1, 1) to cell (3, 5) is shown in Fig. 9-2. The ordered set of elements 
which describe the path is {(, 1), (1,2), (2, 2), (2, 4), (3, 4), (3, 5)}. Of 
course, there are many other directed paths from (1, 1) to (3,5). Note 
that our definition permits a directed path such as that connecting (2, 1) 
with (4, 5) in Fig. 9-3 
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FicunE 9-5 ‚ FicunE 9-6 


DIRECTED Loop: A directed loop in Table 9-1 is a directed path such that 
the first cell in the ordered set is the same as the last cell, and the first branch 
is orthogonal to the last branch. 


A typical directed loop is shown in Fig. 9-4. Two types of loops not 
allowed by our definition are shown in Figs. 9-5, 9-6. The loop in Fig. 9-5 
is not permitted because, according to our definition of a directed path, 
each cell (except the last) must appear only once in the ordered set. In 
Fig. 9-5, (2, 2) appears twice. The loop in Fig. 9-6 is not allowed because, 
if (1, 1) is chosen as the initial cell, the first and last branches are not 
orthogonal. It is still not allowed if any other cell is chosen for the initial 
cell because then it is not true that each pair of adjacent branches is 
orthogonal. Note that we cannot know by looking at a loop such as that 
shown in Fig. 9-4 which element is to be considered the first one. 

From the above definition of a directed loop it is clear that there is always 
an even number of distinct cells in the set which defines the loop. l'urther- 
more, the vectors p;; from A associated with the cells in a loop are linearly 
dependent. To see this it is only necessary to start with any cell in the 
loop, and assign a coefficient of +1 to the selected vector. We move on to 
the next cell (which is in either the same row or column as the first one) 
and assign a coefficient of —1 to that vector. Proceeding in this way, 
alternately assigning coefficients of +1, —1 to the vectors, we obtain 
something like 

pg — prj + Pes — tb Pow — Piv = 0, 
which expresses the linear dependence of the vectors associated with the 
cells in the loop. 

In Section 9-3 we discussed the representation of any vector in terms of 
a set of basis vectors (or equivalently, any vector in a set of linearly de- 
pendent vectors in terms of a subset of linearly independent vectors from 
the given set). We showed that the relation of linear dependence must have 
the form (9-19). But from our definition of a directed loop, we see that the 
cells corresponding to the ordered set of vectors [pi;, p, p, <, P2 Dis} 
form a loop in Table 9-1. 
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9-6 Bases in the transportation tableau. We can now make an interest- 
ing and useful observation. We have demonstrated that the vectors from 
A associated with the cells which form a loop in Table 9-1 are linearly 
dependent. Furthermore, it has been shown that if a given set of vectors 
from A is linearly dependent, then the cells associated with some subset 
of these vectors form a loop in Table 9-1. Hence we conclude that any set 
of vectors for which no loop exists in the corresponding cells of Table 9-1 
is linearly independent. Thus a set of basis vectors from A is a set of m+ 
n — 1 vectors such that no loop exists in the set of cells (Table 9-1) correspond- 
ing to these vectors. Here we have a simple criterion for deciding whether 
any feasible solution given in Table 9-1 is a basic feasible solution. 

We shall introduce a few additional concepts in order to obtain an even 
more vivid geometrical interpretation of the bases and linear dependence 
in Table 9-1, 


SIMPLE DIRECTED PATH: A simple directed path from cell (2,7) to cell (u, v) 
(Table 9-1) is a directed path such that in any row or column of Table 9-1 
there are no more than two cells in the set of cells which defines the path, 


Figure 9-2 shows a simple directed path. If any given directed path 
connecting two elements in Table 9-1 is not simple, it can be reduced to a 
simple path by eliminating some of the cells in the path. To prove this, 
suppose that in row k of Table 9-1 there appear more than two cells of 
the path. Let (k, p), (К, q),-.+, (k, w) be the cells of the path in row k. 
Furthermore, assume that they are written in the order in which they 
appear in the ordered set describing the path. Now we drop all elements 
in the ordered set which lie between (k, p) and (k, w), so that, in the new 
ordered set, (k, p), (k, w) are adjacent cells. We do this for every row and 
column where more than two cells of the path appear. The resulting path 
is a simple path. 

An example will illustrate this procedure: The path shown in Fig. 9-3 
is represented by the ordered set N 


{(2, 1), (4 1), (4, 7), (5, 7), (5, 3), (4, 3), (4, 5)}. 


In row 4, there are 4 elements of the path, and their order in the set is 
(4, 1), (4, 7), (4, 3), (4, 5). According to the above discussion, we drop all 
elements of the path between (4, 1) and (4, 5). This gives the new path 
((2, D, (4, 1), (4, 5)}, which should be a simple path since row 4 was the 
only row or column which contained more than two cells of the original 
path. We see that it is indeed a simple path. 


SIMPLE DIRECTED LOOP: А simple directed loop is a directed loop which has 
no more than two cells in any row or column of Table 9-1. 
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We have shown that any directed path can be reduced to a simple 
directed path. It follows that any directed loop can be reduced to a simple 
directed loop in the same fashion. Clearly, if a loop has one-element in a 
given row or column, it must have at least two elements. A simple loop 
has precisely two elements in every row and column where one element 
of the loop appears. 

Now consider the set consisting of m + n — 1 linearly independent 
basis vectors and one other vector p;;. The set of m + n vectors is linearly 
dependent since p;; сап be written as a linear combination of the basis 
vectors. Hence a loop exists in some subset of the m + n cells (Table 9-1) 
corresponding to these vectors. Furthermore, the loop is unique (except 
for the direction of traversing the loop) since a vector can be expressed in 
terms of a set of basis vectors in only one way. We conclude, therefore, 
that this unique loop must be simple. This follows because we know that if 
there is a directed loop, then there is a simple directed loop. If there were 
а directed loop other than a simple one, the representation of p;; in terms 
of the basis vectors would not be unique. Hence, given any p;; and a set 
of m + n — 1 basis vectors, there always exists a simple directed loop in 
Table 9-1 involving only the cells corresponding to p;; and the basis 
vectors. 


CONNECTED SET OF CELLS: A set of cells in Table 9-1 is said to be con- 
nected if there exists a directed path (and hence a simple directed path) 
involving only cells in the set, that joins any cell in the set to any other cell 
in the set. 


It is very easy to prove that the cells in Table 9-1 corresponding to 
т + n — 1 basis vectors are connected. Assume that there is no path 
joining the cells (s, t), (u, v) corresponding to the basis vectors рё, pZ,, 
which involves only cells that correspond to the remaining basis vectors. 
Then consider the cell (s,v) corresponding to the vector Pss: But Pay is 
linearly dependent on the basis vectors, and hence there is a simple loop 
which involves only (s, v) and the cells corresponding to the basis vectors, 
This leads to a contradiction, because the loop implies that a directed 
path exists which involves only cells corresponding to the basis vectors and 
which joins two cells corresponding to the basis vectors př, and pP; these 
cells are in the same row and column, respectively, as (s, v). If pP, # p8, 
they can be connected by a directed branch. The same is true if рв, # 
p2,. Thus the cells corresponding to a set of basis vectors are connected. 


Basis св: The cells of Table 9-1 corresponding to a given set of 
m +n — 1 basis vectors will be referred to as basis cells of the tableau for 
the given basis. 


TREE: A tree in Table 9-1 is a connected set of cells without loops. 
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FIGURE 9-7, 


Figure 9-8 


Ficure 9-9 


As a result of the above discussion we conclude that a set of basis cells in 
Table 9-1 forms a tree with m +n — 1 cells. Conversely, the vectors cor- 
responding to the cells in any tree containing m + n — 1 elements form a 
basis. A tree containing m + n — 1 cells will be called a basic tree. Observe 
that а basic tree has at least one cell in every row and column of Table 9-1. 
"This follows because the vectors corresponding to the basic cells form a 
basis. If there is no cell in row i, then it is not possible to form a loop 
consisting of one cell in row i and the basic cells. This contradicts the fact 
that we actually have a basis. 


EXAMPLE;"T'wo basic trees for a transportation problem involving four 
origins and six des: aations are shown in Figs. 9-7 and 9-8. It is important 
to note that the branches of a tree may cross each other orthogonally 
provided that there is no cell of the tree at the point of intersection. For 
example, in Fig. 9-9, the branches of a tree cross, but, no loop is formed 
since there are no cells of the set at the points of intersection of the branches. 


—— qn 
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The reader should now be able to see how the tableau form of Table 9-1 
can be used to solve transportation problems in a way that requires con- 
siderably fewer computations than the direct application of the simplex 
method. We shall next present the details of this computational algorithm. 


9-7 The stepping-stone algorithm. Let us suppose that we have a basic 
feasible solution to a given transportation problem such that the m + 
n — 1 basic variables z;; are positive. Then we construct a tableau of the 
form of Table 9-1. We enter the values of the nonzero туу in the appro- 
priate cells and circle these values so that we know which are the basic 
cells. All the costs are also entered into the tableau. To determine whether 
the given basic feasible solution is optimal, we must determine the z;; — cij 
for the pij not in the basis. 

It is easy to compute #4; — суу for the vector pij, i.e., for cell (ї, j). We 
find the unique loop involving (1, j) and the basic cells. Suppose that the 
ordered set of cells is 


[G, 3), @, т), (и,т),..., (в.ш), (8,7), 6,1. 
Then from (9-22), 
2g — су = ср — e pees 08, + су — су. (9-23) 


Starting from cell (i, j), we move around the loop, alternately assigning 
plus and minus signs to the costs. Note that 2; — cj; is the same, no 
matter in which direction we traverse the loop. Enter the value of zij — суу 
in the space reserved for ту (this space is not used since aj; = 0 for this 
cell). In this way, we can evaluate all z; — ci; for vectors pij not in the 
basic solution, and enter them in the tableau. Recall that we are solving a 
minimization problem. Thus, if all zi; — су < 0, the basic feasible solu- 
tion is optimal. 

If one or more of the гу — суу > 0, we know that the value of z can be 
reduced. This follows since we have assumed that the solution has m + 
^ — 1 positive xy, ie, the basic solution is not degenerate. For the 
moment, we shall assume that degeneracy does not occur. Later we shall 
see that it is very easy to handle degeneracy in the transportation tableau. 
The reader might note that we never need worry about an unbounded 
solution in a transportation problem. The absolute minimum cost is zero. 

We are now ready to determine a new basic solution. As in the simplex 
method, we compute 


at — Coe = max(z; — Cj) for 25 — Cg > 0. (9-24) 


This means that, in the simplex method, pa: enters the basis, that is, £s 


292 TRANSPORTATION PROBLEMS [снАР. 9 


becomes positive in the next tableau, and the vector to leave would be 
determined from 4 
min =, $ 2.0. 
ys Yat 

However, the y$? > 0 have the value unity. Hence the variable that is 
driven to zero in the current basic solution is the smallest x2, of those х8, for 
which the coefficient of c2, in (9-23) is +1 when 2,, — cs: appears on the 
left-hand side of the expression. In other words, to find the variable to 
be removed from the current basis, we determine the loop involving (s, t) 
and the basis cells, From the cells for which the coefficient of сд, is +1 
when zs is written in terms of prices in the basis, we choose the smallest 
zB, say 18. 

Then the values of the basic variables for the new basic solution are 
easily computed. "These new values will be denoted by a caret. We have 


2. = 22 (9-252) 
and 
2%, = a ag (9-25b) 


for those x, in the loop involving (s, t) for which cB had a coefficient of +-1. 
Also, 
: 4% = 25 + 22 (9-250) 
for those 22, in the loop involving (s, t) for which св, had a coefficient of — 1. 

Finally, 
£3, 25 (9-25d) 


for those 25, that were not in the loop involving (s,/). Although this 
explanation may seem a little complicated, an example will illustrate that 
the procedure is really quite simple. Noting that the y2? can only take on 
the values +1, 0, we see that the above formulas follow directly from the 
transformation formulas of the simplex method. Ь 

A new tableau is constructed for this new basic solution. The new 
values of the basic variables are entered in this tableau and are circled. 
Again, all the гу — ci; are computed as described above, and the whole 
procedure is repeated. When all z;; — ci; < 0, we have found an optimal 
solution to the transportation problem. 

Here we have developed a very simple algorithm for solving any trans- 
portation problem, given an initial basic feasible solution, This algorithm 
is, in fact, the simplex method. However, because all у are +1, 0, most 
of the work involved in the simplex method can be eliminated since we 
never need to compute the yg. This allows us to solve the problem simply 
by means of addition and subtraction, and furthermore, it enables us to 
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use a tableau involving only m rows and n columns rather than the much 
larger simplex tableau. The above algorithm is sometimes referred to as 
the stepping-stone method; it was introduced by Charnes and Cooper [1]. 
However, Dantzig [8] was the first to find a short-cut method of solving the 
transportation problem and to develop the tableau form of Table 9-1; 
he uses duality to compute the z;; — ci; in a different and more efficient 
way. We shall introduce this additional computational short cut later. 

If the simplex method is directly applied to the transportation problem, 
it is necessary to begin with a basis containing only artificial vectors. 
Fortunately, an initial basic feasible solution can be obtained directly from 
the tableau form of Table 9-1. Artificial vectors are never needed. A 
particularly simple method of determining an initial basic feasible solution 
is the so-called northwest-corner rule, introduced by Charnes and Cooper. 
We begin with cell (1, 1). Set z;; = min (a, bı). At this first step, we 
satisfy either an origin or a destination requirement. If a; > bı, we move 
to cell (1, 2) and set түз = min (a; — bı, b2). On the other hand, if 
b, > ау, we move to cell (2, 1) and set #21 = min (b; — a;, аз). (When 
a, = bj, degeneracy occurs; this will be treated later.) At the second step, 
we satisfy either the second origin or the second destination requirement. 
We continue in this way, satisfying at the kth step either an origin or a 
destination requirement. Ultimately, we obtain a feasible solution. Now 
we note that this method cannot yield more than m + n — 1 positive 
ху because, at each step, we satisfy an origin or a destination requirement. 
After m + n — 1 steps, m + n — 1 of the constraints will be satisfied. 
Since r(A) = m + n — 1, the remaining constraint will be automatically 
satisfied at this point. In the absence of degeneracy, we do not obtain less 
than m 4+ n — 1 positive туу. In this case, it is clear that the resulting 
solution is basic because the method of constructing the solution rules out 
any possibility of loops, that is, at each step, we satisfy a row or column 
constraint and, in constructing the solution, we move down and to the 
right. We never double back, and hence a loop cannot be formed. By 
moving down and to the right we mean that if at step k — 1, we assign 
a positive value to туу, then at step k, we assign a positive value to 23,541 
or туу. At each step, one subscript increases, and the other remains 
unchanged. The following example (Section 9-8) will clarify the above 
exposition. 

The basic feasible solution obtained by means of the northwest-corner 
rule may be far from optimal since the costs were completely ignored. 
Later we shall discuss methods of determining an initial basie feasible 
solution which do take account of the costs. The additional effort spent in 
obtaining a good initial basic solution is worth while because it can con- 
siderably reduce the number of iterations which will be required to find 


an optimal solution. 
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We wish to note that a standard simplex tableau can be constructed 
quite easily from the information contained in a transportation tableau. 
From our knowledge of the basic variables we know which vectors are in 
the basic solution. These m +n — 1 vectors, along with any artificial 
vector, will yield an (m + n)th-order basis matrix. The Zij — cj; are given 
in the transportation tableau. The yf? can be determined very quickly by 
finding the loop which connects cell (2, j) and the basis cells. 

The stepping-stone algorithm permits us to solve transportation prob- 
lems that are too large to lend themselves to direct application of the 
simplex method. For example, on a computer, it is not difficult to solve a 
transportation problem involving 1,000 destinations and 25 origins by 
means of the stepping-stone algorithm or some of its variants. As pre- 
viously noted, no computer is large enough to solve the same problem by 
direct application of the simplex method. The stepping-stone algorithm 
has another property which is very important when a digital computer is 
to be used: It requires only the arithmetic operations of addition and 
subtraction. A digital computer can be.made to operate (in so-called fixed- 
point arithmetic) so that no rounding-off errors occur in addition or sub- 
traction. Thus, the stepping-stone algorithm makes it possible to avoid 
the problem of rounding-off errors which limit the size of the problems 
that can be solved by the simplex method. A transportation problem may 
require an arbitrarily large number of iterations, and no loss of accuracy 
due to rounding-off will occur if fixed-point arithmetic is used,* 


9-8 An example. The general discussion presented above will now be 
applied to a very simple example. Consider a transportation problem 
involving 4 origins and 6 destinations. The origin availabilities, the 
destination requirements, and the costs are given in Tableau 1, Table 9-2. 

An initial basic feasible solution can be obtained by means of the north- 
west-corner rule. We set жү = min (ау, Ьу) = min (50, 30) = 30. Thus 
we have satisfied the requirements of destination 1. We have not used all 
units available at origin 1; hence we set дә = min (a4 — bi, ba) = 
min (20, 50) = 20. Now the first origin constraint is satisfied, Since an 
additional 30 units must be shipped to destination 2, we move to cell (2, 2) 
and set «22 = min (b, — 20, аз) = min (30,40) = 30. Thus the re- 
quirements of destination 2 are satisfied. Ten units are still available at 
origin 2; thus x23 = min (10, 03) = 10. An additional 10 units must be 
shipped to destination 3; set x33 = 10. This leaves 50 units still to be 
shipped from origin 3. We set x34 = 40 = by and satisfy the requirement 
of destination 4. Since 10 units remain to be shipped from origin 3, we set 


* This is not true, however, if floating-point arithmetic is used. In this case, 
rounding-off errors can occur. 


e 
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хз = 10. The requirement at destination 5 is 30; set x4; = 20. This 
leaves 11 units to be shipped from origin 5, which is precisely the number of 
units required at destination 6. We circle the 2;;-values just obtained and 
note that they are 9 = m - n — 1 in number. Furthermore, these cells 
do not form a loop. Thus we have a basic feasible solution. All other z;; 
are zero. We do not fill in these zeros. This initial basic feasible solution is 
given in Table 9-2. 

The next task is to compute the z;; — cij. Let us illustrate the procedure 
by calculating 241 — сал. First, we find the unique loop involving (4, 1) 
and the basis cells. This loop, shown in Table 9-2, is indeed unique and 
simple. For this special case, (9-23) becomes 


241 — C41 = C45 — Cas + C33 — C23 + C22 — C12 F C11 — ба, 


or 
244—441 = 2—44+4—242—-142—4= —1, 


The value —1 is now entered in cell (4, 1). We use boldface type for the 
zij — су to avoid confusion with the costs. In the actual working of 
problems by hand, it is often helpful to use different colors for the cij, 
Zij — су, and the basic variables. Note that we obtain the same value of 
241 — c41, regardless of the direction in which we traverse the loop, i.e., 


241 — баа = сүү — C12 + C22 — Сӧз + Саз — C35 + Cis — ба. 


All the other z;; — c; are computed in the same way. We do not compute 
the zi; — суу for the basic cells because we know that zij — су = 0 for 
these cells. However, zero values of the z;; — cij for nonbasie cells are 


written in. 

Not ай z;; — с; € 0, so the initial basic feasible solution is not optimal, 
The largest z;; — сг; iS 236 — C36 = 3. At the next stage, зв becomes 
positive (pas enters the basis in the simplex method). To determine the 
variable to leave the basis we find the loop involving (3, 6) and the basis 


cells. "This loop is the dashed line shown in Table 9-2. We note that 
236 — 36 77.685 — C45 + Сав — C36- 


Only c35 and сав have plus signs; hence only these basic cells give у = 41. 
Now x35 = 10, тав = 11. The smallest of these is x3; = 10. For the new 


basic solution, 
236 = 275 “= 10, 245 = 20 + 235 = 30, #46 = 11 — 235 = 1. 


All basic variables not in the loop are unchanged in value. These results 
follow from (9-25). 
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Without worrying about the у, we can easily see how the z;; should 
change when a given variable is to be brought into the solution, Examina- 
tion of the loop involving cell (3,6) and the basis cells shows that to 
satisfy origin constraint 3, x35 must be decreased by @ if £36 is increased 
by 6. In order to satisfy destination constraint 5, £45 must be increased 
by 6, and to satisfy origin constraint 4, 246 must be decreased by 6. The 
largest permissible value of @ is that which first drives one of the basic 
variables to zero. As soon as one knows the cells which are in the basic 
solution, the values of the basic variables can be easily determined by 
satisfying the constraints. No transformation formula is needed. 

The new basic feasible solution is given in Table 9-3. It should be 
observed that it is very easy to obtain the new basic feasible solution. 
The bulk of the work in the stepping-stone algorithm comes in evaluating 
the zij — c; Having found a new basic feasible solution, we must deter- 
mine the new z;; — cij- We can evaluate these by the loop method just 
as we did for the first tableau. These new z;; — c;; are given in Table 9-3. 
On occasions, it can be a little tricky to find the required loop, especially, 
if a fairly large problem is being solved. (In Table 9-3, we have drawn 
the loop needed to evaluate 225 — cag, for example.) 

The remaining tableaux are presented in Tables 9-4 through 9-6. In 
Table 9-3, the choice of the vector to enter the basis is not uniquely 
determined by (9-24). Cell (4, 3) was chosen arbitrarily to enter the basis. 
The tableaux do not contain the value of the objective function. Therefore, 
we must, compute the minimum value of z from the last tableau. We see 
that 


min г = 2(20) + 3(10) + 1(30) + 2(20) + 2(20) + 2(39) + 1(1) 
-F 2(30) + 1(11) = 330. 


'This can be compared with the cost for the initial basie feasible solution, 
which is 


z = 2(30) + 1(20) + 2(30) + 2(10) + 4(10) + 2(40) + 4(10) 
+ 2(20) + 2(11) = 382. 


The reader may find it instructive to draw in the branches of the basic 
trees in Tables 9-2 through 9-6. Some of these have an interesting struc- 
ture. 

Note: To compute the zi; — cij, for all tableaux after the first, one 
could use the standard transformation formula of the simplex method. 
For the transportation problem, this transformation formula is 

ar 
ys 


15 — 64 = Zij — eg — 
U^ 


(Zee — Cat), (9-26) 
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TABLE 9-2 — TABLEAU 1 
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TABLE 9-4 — TABLEAU 3 


а 
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TABLE 9-6 — TABLEAU 5 


Dı Dz Da Da Ds ai 
Е И ас mA 
o| Qo) e 0 50 
_3| аа 
02 0 (20) (20), | -2 0 40 
3 5| 4] 2| а] 
05 ae = ii 60 
4] re cs ара 
O| —2 x i (5 31 
b; | 30 50 20 40 30 181 


where yf? = +1. The value of уу can be 0, +1. This method is easier 
than the loop method since, at worst, we only need to add or subtract two 
numbers. We must, however, determine yf. This requires that we find 
the loop involving (i, j) and the basis cells in the previous tableau (not in 
the new tableau where we enter the values of the zij — cij). 

Let us illustrate the computation of 241 — ca: for tableau 2 from the 
information in tableau 1. From (9-26) we see that 


35 35 
241 — Cay = 241 — Cav 8041 = —1 — Зуя, 


since г, — сы = 236 — Cas = 3. To determine yj}, we go back to Table 
9-2 and find the loop (plotted) connecting (4, 1) and the basis elements. 
We see that 335 = —1, so that 241 — Са = —1 + 3 = 2; this is the 
value shown in Table 9-3. The transformation formula (9-26) is seldom 
used to find the z;; — cij, because it is simpler to work with a single 
tableau than with two tableaux. 


9-9 Degeneracy and the transportation tableau. A feasible solution to а, 
transportation problem is degenerate if less than m + n — 1 of the zi; 
are positive. Degeneracy may be encountered in the process of determining 
the initial basic feasible solution or at some subsequent iteration. From 
the practical point of view, degeneracy does not cause any difficulties, just. 
as it did not cause any real difficulties in the simplex method. No trans- 
portation problem has ever been known to cycle. In the simplex method, 
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the degeneracy problem can be eliminated by the use of a perturbation 
method. The same is true for the transportation problem; however, a 
much simpler perturbation method can be used. 

Let us begin our discussion by showing how degeneracy can be handled 
in practice, without introducing some technique which ensures that cycling 
will not occur. If the method used to provide an initial solution yields à 
feasible solution with h < m +n — 1 of the Tjj > 0, we encounter 
degeneracy at the outset. The cells associated with the positive z;; do not 
form a basic tree. To obtain an initial basic solution and a basic tree in the 
tableau, we must add m + n — 1 — h additional cells at a zero level. 
We choose the cells to be added such that the resulting m -+ n — 1 
cells form a basic tree. We enter a zero into the cells added and circle these 
zeros to indicate that they are part of the initial basic solution. In manual 
computations, it is very easy to choose cells which will yield a basic tree 
(there may be, of course, many different combinations). In the next 
section, we shall present a simple method of automatically determining the 
cells to be added at a zero level, which is useful when the problem is to be 
solved on a digital computer, Having determined a basic solution, we 
proceed in the usual way. In this case a variable can enter and leave the 
solution at a zero level. 


ExAMPLE: Consider the tableau in Table 9-7. Costs are omitted since 
they are not relevant to the discussion. Using the northwest-corner rule 
to find an initial feasible solution, we obtain the results shown in Table 9-7. 
Here we have only 8 positive z;;, although m + n — 1 = 10. It will be 
noted that the set of cells corresponding to the positive 7; is not connected. 


TABLE 9-7 

Di 

«® 
02 
Оз 
х A 
05 

b; 20 
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Two more cells are needed to obtain a basie tree. Clearly, if we add cells 
(2, 3), (4, 5) (dashed circles), a basic tree results. The value zero is entered 
into these cells, and we have a basic (degenerate) feasible solution. The 
annexed cells were by no means the only two cells which could have been 
added to yield a basic tree. 

If the z;; — c;; are such that cell (2, 4) should appear in the basie tree 
at the next iteration, it will be observed that x24 will enter the basic 
solution at a zero level. The values of the variables in the basic solution 
remain unchanged. We only hawe to replace cell (2,3) by cell (2,4) to 
obtain a new basic tree. $ 


Degeneracy сап also appear at some later iteration if there is а tie for 
the variable to leave the basic solution. As in the simplex method, we can 
choose arbitrarily any one of the tied variables as the variable to leave the 
basis. At the next stage, the variables that were tied with the removed 
variable will be at a zero level.. However, we keep these variables in the 
basic solution (i.e., they remain circled) and proceed as usual. 


ExaAMPLE: Suppose that in the problem set forth in Table 9-2, we change 
the availability at origin 4 to 30 and the requirement at destination 6 to 10. 
For this new problem, the northwest-corner rule yields the initial basic 
feasible solution shown in Table 9-8. 


The z;; — суу are the same as in Table 9-2, so #36 enters the basic solu- 
tion at the next stage. Now, however, there is a tie for the variable to be 
removed. Either 235 or z4s can be replaced by га. If we arbitrarily decide 
to replace 235, the new basic solution is that shown in Table 9-9. One of 
the basie variables is now zero, and degeneracy has appeared. Again, the 


TABLE 9-8 


Dy Рә Da Ds Ds Do а 
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20 40 30 10 180 


Zij — сіу are the same às in Table 9-3. If x43 is chosen to enter the basis, 
it enters at a zero level. 


Let us now turn our attention to the study of a method which completely 
eliminates the possibility of degeneracy. We begin our discussion by show- 
ing that if degeneracy occurs at any stage, then at least one proper subset I 
of the a; and at least one proper subset J of the b; exist such that 37;e; a; = 
Dies bj, the summations being taken over the а;, b; in the appropriate 
subsets. Before presenting the proof, we wish to note that this was indeed 
the case in the preceding examples where degeneracy occurred. In Table 
9-7, а + аз = bi + be = 50, und ag = ‘bs + bg. In addition to these 
subsets, there is a number of other proper subsets in Table 9-7 such that 
Xa; = Y); In Table 9-8, a, + a4 = bı + bo = 80, az + аз = ba + 
by + bo, ete. 

Suppose then, that we have a degenerate basic feasible solution. With- 
out loss in generality, we can assume n 2 m. Thus for the basic feasible 
solution, there will be at least one column of Table 9-1 which contains 
only one positive x;;. Otherwise there would have to be at least 2n posi- 
tive values of z;; in the basic solution, with the result that 2n <m+ 
^ — lorn < m — 1, which contradicts our assumption. For any such 
Tij We must have z;; = bj. Now we cross off column 2, replace a; by 
4j) = a; — bj, and consider the new tableau containing m rows and. 
n — n — 1 columns. For this new tableau, n® > m or m > n" so 
that there will be at least one row or column which contains only one 
positive жуу. Suppose zy, is the only positive Zu; in row u. We cross off 
row и and replace b, by b® = b, — ay. 

After k steps we are left with a tableau containing a total of m + n — k 
rows and columns. Furthermore, the аў, МЮ, which are the origin avail- 


> 
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abilities and destination requirements for this tableau, will have the form 
a? = a; — Ж Day, |)? = b; Eak УЬ, . (9-27) 


the sumntations being taken over some subsets of the crossed-off rows and 
columns. When we reach one of the basic variables for which ts = 0, 
either the af or b? will be zero (since z; is the only basic variable in row s 
or column ё). In both cases, Eq. (9-27) leads to the conclusion that Xa; = 
Xb; for some proper subsets of a;, b; in the original tableau. 


ExAMPLE: For the degenerate basic solution given in Table 9-7, x1, is 
the only basie variable in column 1. We cross off column 1 and replace a; 
by a? = a, — b, in the new tableau. Now 7,» is the only basic element 
in row 1. We cross off row 1 and replace bz by 02) = bz — af? = bz — 
a; + bı. Then zs» is the only basic element in сой 1 of the new tableau 
(column 2 of Table 9-7). We cross off this column and replace az by а = 
аз — DY = az — ba +a, — bi = 0. But 253 = 0 is the only element 
in this row. Since a? = 0, everything works out properly. However, 
a = 0 implies that a; + аз = bı + bs, as mentioned above. 


This result proves that degeneracy can never appear in a transportation 
problem unless the sum of some proper subset of the a; is equal to the sum 
of some proper subset of the b;. Imagine now that we perturb the problem 
slightly, that is, the origin availabilities are a; = a; + € ? = 1,...,m, 
and the destination requirements are bj = bj, j = 1,...,n — 1, = 
b, + me. We note that 


Datos Sy bot bat me 


tel 
so that the problem still has a solution. We shall demonstrate that there 
exists an eg > Osuch thatforalle,0 < є < €o, degeneracy will never occur. 

In the original problem, there is only a finite number of partial sums 
of the а;, bj. Consider all sets of partial sums containing the a}, bj. These 
partial sums will not satisfy Xa = Xb; for all є since € will never cancel 
out in such an expression. They are linear i in e, and in each case we can 
solve for the e such that Da; = Xb}. If we choose eo to be the smallest 
of the resulting positive e’s*, then for any €, 0 < € < €o, it can never be 
true that any partial sums Bi asy Xj = УЫ; Hence degeneracy can 
never appear in the perturbed problem. 

When solving a transportation problem by hand, wescan easily carry 
along the єз. It is never necessary to determine € explicitly. There is no 
need to introduce the e’s until a tie occurs; then they can be introduced to 
break the tie. Naturally, when an optimal solution is found, we drop the 
€'s to obtain the solution to the original problem. 


* Note that if no e > буер can be arbitrarily large. 


* 
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EXAMPLE: If the e’s are introduced into Table 9-7, we obtain Table 9-10. 


If Уа; = 3b; for one or more partial sums, one should be careful not 
to infer that degeneracy will occur. In the example worked out in Tables 
9-2 through 9-6, it will be noted that a + аз = be + bg + bs, a2 = ba, 
ete. However, degeneracy did not appear in the solution of this 
example. 


9-10 Determination of an initial basic feasible solution. The northwest- 
corner rule for determining an initial basic feasible solution to a transporta- 
tion problem has already been discussed. Now we shall present some 
other methods which often yield a result which is much closer to an optimal 
solution than that obtained by the northwest-corner rule. As suggested 
before, it is worth while to spend some time finding a “good” initial solu- 
tion because it can considerably reduce the total number of iterations 
required to reach an optimal solution. 

Most of the methods for determining an initial basic feasible solution 
assign a positive value to one variable and, at the same time, satisfy either 

„а row or column constraint at each step. We shall now prove that any 
“procedure for determining a feasible solution which assigns a positive 
value to one vafiable and satisfies either a row or column constraint at 
each step will automatically yield a basic feasible solution, and in the 
absence of degeneracy, the resulting cells will form a basic tree. Note 
first of all that such a technique cannot give more than m + n — 1 posi- 
tive variables since, after m +n — 1 steps, т + n — 1 of the constraints 
will be satisfied, and the remaining constraint is automatically satisfied. 


> 
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All that remains is to demonstrate that a loop cannot be formed. Sup- 
pose that at step k a positive value is assigned to z;;, and cell (7, J), together 
with some or all of the k — 1 cells corresponding to previously determined 
positive values of the variables, forms a loop. Let the loop be described 
by the ordered set. 


(#7), @ т), (& n), ..., @, м), OD, (62 


Since we are allowed to assign a positive value to z;;, it must be true that 
by assigning a positive value to ту, we have satisfied the column constraint 
r rather than the row constraint 7. This means that a positive value must, 
have been assigned to tsr before it was assigned to =;,, and at that step, 
the row constraint s had been satisfied. Proceeding in this way, we con- 
clude that the column constraint j was satisfied by assigning a positive 
value to a,j. However, we are now assigning a positive value to z;;, which 
contradicts the fact that the constraint j has been satisfied. Thus no 
loop can be formed, and a basic solution is obtained. In the absence of 
degeneracy, the basic cells will be m + n — 1 in number and will form a 
basic tree, When degeneracy occurs, one or more additional cells must be 
included at a zero level to yield a basic tree. 

When degeneracy occurs, an automatic procedure for adding cells at a 
zero level in order to obtain a basic tree follows immediately if the initial 
solution is found by assigning at each step a value to one variable such that 
either a row or column constraint is satisfied. For example, suppose that 
at step К, a row and a column constraint are satisfied simultaneously. This 
means that degeneracy has appeared. At this point, imagine that either the 
row or the column constraint satisfied at step k is perturbed by increasing 
its requirement by e. Then continue the process of finding the initial 
solution. The above proof shows that the resulting solution will be basic, 
and when the e’s are set to zero, a degenerate basic solution is found. It is 
really unnecessary to introduce the e's explicitly. The important thing is 
that both a row and a column are never dropped simultaneously.* This 
observation is valuable when the technique is to be used with digital 
computers, since it eliminates the necessity of introducing numerical 
values for the e's. 

We shall now present four methods for obtaining an initial basic feasible 
solution. All make some use of the costs. In the following, whenever we 
say, at any one step, “cross off” a row or column, we mean that no cells 
from that row or column ean be chosen for basis cells at a later step in the 
determination of an initial basic feasible solution. 


* Interestingly enough, an entire paper [4] has been devoted to explaining this 
simple procedure for adding cells at a zero level in order to obtain an initial 
degenerate basic feasible solution. 
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TABLE 9-11 


E at seer pa 
ДКО 
on | 
EE Ln e, 


(1) Column minima: Beginning with column 1 of the tableau, choose the 
minimum cost in this column. Suppose that it occurs in row r. Then set 
2,1 == min (ar, b1). If х,у == by, cross off column 1 and move to column 2. 
Ifz,, = а,, cross off row r from the tableau, and choose the next lowest cost 
in column 1. Assume that it occurs in row s. Set 1,1 = min (as, b, — a,). 
Continue in this way until the requirement at the first destination is satis- 
fied. If the minimum cost is not unique, select any one of the minima. 
When the requirement of column 1 is satisfied, cross off column 1 and 
repeat the above procedure for column 2. Continue until the requirement 
of column n is satisfied. 

In the event that a row constraint and a column constraint, say column k, 
are satisfied simultaneously, cross off only the row. "Then move to the cell 
in column k having the next lowest cost. Assign a value of zero to this cell 
and assume it to be in the basie solution. Now cross off column k and 
move to column $+ 1. This will yield a degenerate basic feasible 
solution. 

If this procedure is used to obtain an initial basic solution for the example 
solved in Tables 9-2 through 9-6, we obtain Table 9-11 for the first 
tableau provided that in columns 2, 3 the row with the lowest. index is 
chosen when the minimum cost is not unique. 

(2) Row minima: Beginning with row 1, we choose the minimum cost in 
this row. Suppose that it occurs in column r. Set Tir = min (ay, b,). If 
21, = а, cross off row 1 and move to row 2, If Zir = br, cross off column r 
and determine the next lowest cost in row 1. Assume it occurs in column s. 
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TABLE 9-12 
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Set z,,— min (a; — by, bs). Continue in this way until the first row 
constraint is satisfied. When the requirement of the first row is satisfied, 
cross off row 1 and repeat the above procedure for row 2. Continue until 
the row constraint m is satisfied. Whenever a minimum cost is not unique, 
make an arbitrary choice among the minima. 

In the event that a row constraint, say row k, and a column constraint 
are satisfied simultaneously, cross off only the column. Then find the next 
lowest cost in row k, and insert this cell into the solution at a zero level. 
Then cross off row k, and move on to row k + 1. 

If this technique is used to obtain a first solution for the problem of 
Table 9-11, Table 9-12 results. Note that degeneracy appears. According 
to our rule, either cell (1, 1) or (1, 5) could have been added at a zero 
level. We chose (1, 1). 

(3) Matrix minima: Determine the smallest cost in the entire tableau. 
Suppose this occurs for cell (i,j). Set xij = min (ai, bj). Then cross off 
either row i or column j, depending on’ which requirement is satisfied. If 
xij = ai, decrease bj by а, and if zi; = bj, decrease a; by bj. Repeat the 
process for the resulting tableau. Whenever the minimum cost is not unique, 
make an arbitrary choice among the minima. As usual, if a row and a 
column constraint are satisfied simultaneously, cross off only the row or 
the column, not both. 

This method yields the initial solution (shown in Table 9-13) for the 
example under consideration if, in the absence of a unique minimum, we 
choose the cell for which 7 + j is smallest. Here again degeneracy appears. 
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(4) Vogel's method: This technique has been suggested by Vogel [9]. 
For each row, find the lowest cost cij and the next lowest. cost су in that row. 
Compute cj, — cij. In this way, m numbers are obtained. Proceed in 
precisely the same way for each of the columns and obtain n more numbers. 
Choose the largest of these m + n differences. Suppose that the largest of 
these numbers was associated with the difference in column j. Furthermore, 
let cell (т, 7) contain the lowest cost in columnj. Then set tij = min (a;, bj). 
Cross off either row ? or column J, depending on which requirement is 
satisfied, and repeat the whole process for the resulting tableau. As usual, 
when the maximum difference is not unique, an arbitrary choice can be 
made, and if a row and a column constraint are satisfied simultaneously, 
cross off only the row or the column, not both. 

For our example, this method yields the solution shown in Table 9-14. 
It is convenient to, list the row differences in a column to the right of the 
tableau and the column differences in a row at the bottom of the tableau. 
"The differences shown in the difference row and column are those for the 
first step, i.e., those which are to be used in selecting the first basis cell. 
Here we have the worst possible case; every difference has the same value. 
The tie is resolved by choosing the cell with the smallest value of т + j. 
At each step, a new set of differences must be computed. 

Many other techniques for determining an initial solution might have 
been discussed. However, those presented above are most commonly 
used for hand computations and for digital computers, It is by no means 
established that any one of the methods is better than the others. In fact, 


سی نی ت ت и ——— RAPI‏ 
ЧА‏ س س ما ھت ہک —— 


9-11] COMPUTING Zij — Cij; DUALITY 309 


TABLE 9-14 
D3 Ds Ds De ai 
3 3] 2 5] Ye] 
50|1 
|2 а] 3 а] 
40 |1 
4 
60|1 
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for our particular example, it is not even certain that these methods are 
more efficient than the northwest-corner rule. However, the northwest- 
corner rule looked as good as it did only because we arranged the data so 
that the lowest prices were near the main diagonal. To decide which 
method for determining on initial basic feasible solution leads to the 
smallest number of iterations for our example, it would be necessary to 
solve the problem in each case. This task is left to the problems. 


9-11 Alternative procedure for computing z; — cj; duality. In. his 
original work on the transportation problem, Dantzig [8] presented a 
method for evaluating the z;; — ci; which is simpler than the stepping- 
stone technique. Let cB, cB, cB, ..., Che, су be the m +n — 1 prices 
corresponding to the variables in any basic feasible solution to a trans- 
portation problem with m origins and n destinations. Now suppose that 
we write t 
uz + Ur Cir, 

B 
Ug + U, = Car, 
Ug + Ve = с (9-28) 


B 
Uw + Us = Cus, 


B 
Uy + 05 = Сој. 
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Consider (9-28) to be a set of m + n — 1 simultaneous linear equations in 
the m + n variables us, vs. If we write (9-28) in matrix form, numbering 
the variables so that the ua starting with the lowest subscript come first 
and are followed by the vg arranged in the same order, it will be observed 
that the rows of the matrix of the coefficients are simply the vectors p, in 
the basic solution. Thus the rank of the matrix of the coefficients is 
т - п = 1. We can assign an arbitrary value to one of the Ua, Vg and 
solve uniquely for the remaining m + n — 1 variables. These equations 
are extremely easy to solve since they can be solved sequentially. If, for 
example, we set u; = 0, then v, = cË, uy = cB — cf, ete. Note that, 
in solving the above set of equations, we obtain m, ua and n, va so that 
there is a и, corresponding to each row and а va corresponding to each 
column of the tableau. 
The introduction of the ua, vs has an interesting result: For any cell 
(2), 
2j — су = Ui + vj с, (9-29) 


and the savings in computational effort become apparent. As soon as the 
Ча, їз are determined, all the z;; — суу can be found by adding two numbers 
and subtracting a third. The proof is trivial. We can write 


Bei бу = Ci, rd оф + 6o с; 
but by (9-28), 


Big оу = шщ De lg — Vp dou руне vj — cij 


= Uit vj = с. (9-30) 


For hand computations, it is convenient to augment the tableau by an 
additional column which gives the ua and an additional row which gives the 
vg. Then the zij — cij are determined as follows: We move in row 7 to the 
u-column to find и;, and in column j to the v-row to find v;; the value of 
Zij — ci; is given by (9-29). 


ExawPLE: Let us use the ц, v; to compute the Zij — ¢;; in Table 9-6. 
The prices in the basis аге сүү, e31, C12; C22, C23, C34, C44, C45, C3. The set 
of simultaneous equations to be solved is 
| 

Uuckuu-o0i:-2, wu фо = с = 1, Ug + V2 = соз = 2, 


2, 


1 


üz + 0з = Соз = 2, ugt vy = c31 = 3, Us + 04 = C34 


из + 06 = c36 = l, u4 4-04 = си иа +05 = cas = 2. 


T. 


Setting иу = 0, wefindv, = 2,0, = 1, иә = 1,03 = l ug = 1,04 = 1 
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TaBLe 9-15 

а Ui 
50 | 0 
40| 1 
60 | 1 
31 | 0 

bj 30 50 20 40 30 1n |181 

vj 2 1 1 1 2 0 


u4 = 0, 05 = 2, ve = 0. We can then construct the tableau given in 
Table 9-15. For example: To compute z41 — c41, we note that u4 = 0, 
v; = 2, са = 4, and hence 24 —¢41 = 0 +2 — 4 = —2. The 
(zij — cij)-values in Table 9-15 check with those obtained in Table 9-6. 


After using the uv-method (as we shall call the present technique*) on 
several tableaux, the reader will be convinced that it involves much less 
work than the stepping-stone technique, especially for large tableaux. 
Furthermore, the chances of making a numerical mistake are considerably 
reduced. Almost all digital computer codes for solving transportation 
problems use the w-method to evaluate the zij — cij rather than the 
stepping-stone technique. 

The u;, v; introduced above are, of course, nothing but the variables for 
the dual of the transportation problem. Taking the dual of (9-2), (9-3), 
and (9-4), we can write 


uy foe сар TH 1,2. m, Jj-1l...m 
m n 
max Z = 5T ағ: + >), 
i=l jæi 


* This method is sometimes also referred to as the MODI method. 
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where the 1;, v; are unrestricted in sign. The dual problem has mn con- 
straints and m 4- n variables. 

The values of the dual variables obtained from (9-28) do not satisfy the 
dual constraints unless the solution to the primal is optimal. Note that if 
Tij > 0, then wu; + vj = суу, so that the complementary slackness condi- 
tion is always satisfied. The dual variables are not uniquely determined 
from these m -+ n — 1 equations (9-28) because one of the m + n con- 
straints of the primal is redundant, and we have a dual variable which 
corresponds to this redundant constraint. 


9-12 Inequalities in the constraints of a transportation problem. Let 
us return to the formulation of the transportation problem given in 
Chapter 1. It had the form 


n=l 
BS ay i—l...,m, 
j=1 
х 
2524 = by, jl age — 1, (9-31) 


£j 20, a, 


minz — p» Citi. 
#1 

For a reason that will soon become clear, we here use n — 1 rather than n. 
The first m constraints now contain a S sign rather than an equality 
Sign. Physically, this simply means that more units may be available at 
the origins than are required at the destinations. 

The inequalities can be converted to equalities by the addition of m 
slack variables, These slack variables will be written £in, 4 = 1,..., m. 
Then the constraints become 


n1 
Tid zi = a i-—1,..,m, (9-32) 


йб petisse (9-33), 


i=l 
Sum (9-32) over i and subtract from the result the sum of (9-33) over j. 
This gives 
m m n—1 
DD Sy 3 b = b. (9-34) 
ї=1 i=l j=1 s 


Here we have the interesting and intuitively obvious result that the total 
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slack, i.e. the sum of the slack variables, remains constant and is the dif- 
ference, denoted by bn, between the origin availabilities and the destination 
requirements. Note that if we annex (9-34) to (9-33), we have reduced the 
problem expressed by (9-31) to the problem formulated in Eqs. (9-2) 
through (9-4); i.e., to the problem considered above. To construct the 
tableau, we simply add one more column, i.e., an additional destination 
| for the slack. Intuitively, this approach is to be expected since the units 
not shipped can be considered to be shipped to ourselves at no cost. As 

usual, the cost Cin associated with the slack variable Tin is zero. 
Instead of Eq. (9-31) we shall now consider the following meaningful 


problem: 
п 
Уа i=l nam — 1, 
j=l 
m—1 
Do tatis ЖЕКШЕ. 
im (9-35) 
vg Z0, alij, 
maxz — ES Cijtij. 
1j 
Introduce the surplus variables x,,;, j = 1,...,n and note that 
n n m—1 
= Уан У) — aS a, <0. (9-36) 
51 j=l i=l 


The constraints of (9-35) can therefore be converted into the set of equa- 
tions 


А 
Уту = an t= 1,...,m—1, 


j=1 
Уаз 10 (9-37) 
j=l 
› т—1 3 
Tij — mj = bp ў=1,...,т. 
i=l 


Here, a new difficulty arises because the coefficients of the surplus 
variables are —1 instead of 1, Again, it is easy to show that all minors of 
the matrix of the coefficients have the value 4-1, 0. The computational 
method is precisely the same as before except that we use 25; = —Um — vj 
for computing 2m; — cj. This follows immediately from the dual. To 
solve this problem, we add one more row to the tableau, ie., an 
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additional origin containing the negative of the total surplus. A detailed 
discussion of the above example is left for Problem 9-27. 

We wish to note that, provided all costs are positive, then in the optimal 
solution to a problem of the form 


5 
TERS AEE doors 
j=l 


m 
mpi s = бауу 
i=l (9-38) 


Tj 2 0, alij, 


max or min z = xS Cii, 
ij 


strict equalities will hold (a) in the destination constraints if z is to be 
minimized and (b) in the origin constraints if z is to be maximized. Physi- 
cally, this means that if we are minimizing the cost, no more will be 
shipped than necessary, and if we are maximizing z, as much will be 
shipped as possible. It is easy to prove these facts rigorously; the details 
are left for Problems 9-24 and 9-25. 

Finally, we might note that once a transportation problem has been 
cast into the form (9-2) through (9-4), we can replace each ci; by cij + А 
for any constant А without changing the z;; which yield an optimal solu- 
tion. This ean be done because the substitution of с;; + for each суу 
changes the value of the objective function merely by the constant 
NDF ai 


9-13 Generalized transportation problems. Consider a linear program- 
ming problem of the following form: We wish to find 2492 0,z: > 0, 
which satisfy the constraints 


Dodi onis  a(»0, i-—1..,m, (9-39) 
j=l 
Е 
Уц 210 а. (9-40) 


i=l 
and'which maximize or minimize the linear form 


zig. (9-41) 
i 
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The z,; can be considered to be slack or surplus variables. This linear 
programming problem differs from the transportation problem in that the 
coefficients of the xj; in (9-39) are the arbitrary numbers d;; rather than 
unity. Problems of this type arise in practice in a variety of applications. 
One particular example (to be considered in Chapter 12) is that of ma- 
chine assignments. 

Clearly, the simplex method can be used to solve linenr programming 
problems of the form (9-39) through (9-41). However, the structure of 
these problems is so similar to that of the transportation problem that 
we should be able to devise an algorithm for their solution which requires 
less work than the simplex method. This can be done, and it will be 
shown how the w-algorithm for the transportation problem can be modi- 
fied to solve such problems. We shall call (9-39) through (9-41) a gen- 
eralized transportation problem. The constraints (9-39) arise frequently 
because of limitations of the various available resources. Hence we shall 
call them resource constraints. The constraints (9-40) often specify the 
requirements, and therefore we shall refer to them as requirement con- 
straints. 

There are several important differences between transportation prob- 
lems and generalized transportation problems. First of all, the rank of 
the matrix of the coefficients of the z;; in (9-39) and (9-40) is, in general, 
m - n rather than m + n — 1, i.e., all constraints are independent. In 
the discussions which follow, we shall assume that the coefficient matrix 
has rank m + n. Furthermore, the у for the generalized transportation 
problem can take on values different from +1,0. This in turn means 
that division eannot be eliminated in the generalized transportation 
problem. Consequently, the integrability property of optimal basic solu- 
tions which held for transportation problems does not hold for generalized 
transportation problems. An optimal basic solution may involve non- 
integral values of the aj, even though the а;, bj are integers. 

If we arrange the variables and constraints for the generalized trans- 
portation problem as in (9-5) and (9-6), the activity vector pi; correspond- 
ing to xy is 

ру = Че: + eni (9-42) 


and the activity vector corresponding to ta; is +€; 
Denoting by pë, the m + n activity vectors in any given basic solution, 
we can write any pi; аз 


ру = У) pees (9-43) 
mI 


and if B is the basis matrix containing the p%,, then pi; = By;j. When 
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z in (9-41) is to be minimized, the dual of (9-39) through (9-41) is 


Aiur hey сиро) лт, OF = Aw. jn, (9-44) 


su; < 0, i= l. h (0—15) 
т n 

max Z = Qi аш + 25 Ьу. (9-46) 
ici j=l 


The constraints (9-45) come from the slack or surplus variables; a plus 

sign holds for slack variables and a minus sign for surplus variables. 
For any basic feasible solution to the primal, denote by cB, the prices 

in the primal basis. Then consider the m, ua and n, vs which are the 


solutions to the set of m + n equations 


Bn 
daga + їз = соз, 


nti 9 
ERN | m + n equations. (9-47) 


An equation of the form u. = 0 appears if the slack or surplus variable 
Xsa is in the primal basis. When the ta, va are arranged in the proper order 
into a row vector (u, v) = (u1,..., Um t1... „ Un), then (u, v) = cB.. 
Hence, once the ua, їз are known, we can immediately compute the 
Zij — суу for the corresponding basic solution to the primal from 


Zij = су = CB pig — су = (U, Pi — су = diu; + vj — су, all dj, 
and (9-48) 


Zsi — Cai = (u,v)(-e; = и; (slack variable), —u; (surplus variable). 


9-14 Solving the generalized transportation problem. We are now in à 
position to solve the generalized transportation problem. The tableau 
is the same as that used in solving the transportation problem (see Table 
9-16). Now, however, we must include dij as well as су; in each cell (i, J). 
We denote row i by R; and column j by P;. It will be noted that a; is 
obtained for R; if we multiply z;; by d;; and sum across the row. When 
we sum the z;; in ^j, then 0; is obtained. In this tableau, however, the 
sum of the variables in the slack or surplus column has no meaning. 
For a generalized transportation problem, the total slack or surplus does 
not need to remain constant. Furthermore, it need not be true that 
Xa; = Xb;. The slack or surplus variables enter only when we sum 
across rows in the way indicated above. 

An initial basic feasible solution for the above problem can be obtained 
in almost the same way as for transportation problems. Any unique 
feasible solution involving exactly m + n positive z;; will be a basic 
feasible solution -(see Chapter 2). All the methods suggested in Section 
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TABLE 9-16 
Slack 
or 
Pi Рә P Surplus a; и; 
C11 
Rı E 
dıı 
C21 
Ra 121 
421 
Cml 
Rm Tml 
dmi 
bj bi 
vj vi 


9-10 can be used to obtain a basic feasible solution. There is only one 
difference: When considering the resource constraints, we must remember 
that d;jz;; rather than хуу is the amount used of resource i, Of course, 
we can always obtain a basic feasible solution (if there is one) by adding 
artificial variables. Problem 9-35 asks the reader to show how this can 
be done within the framework of the tableau format of Table 9-16. For 
the generalized transportation problem, it need not be true that the cells 
(in Table 9-16) corresponding to a basic solution form a tree. It is quite 
possible that a loop exists in the set of basic cells. Loops no longer imply 
that the vectors in the loop are linearly dependent (why?). F urthermore, 
it is not necessarily true that the cells for a basic solution yield a con- 
nected graph. 

After a basic feasible solution is obtained, the z;; — c;; must be com- 
puted to determine whether this solution is optimal. First we compute 
the wa, va by solving the set of equations (9-47); then we use (9-48) to 
find the z;; — суу. The equations (9-47) are fairly easy to solve, and 
they can often be solved sequentially just as they can for transportation 
problems. l'or generalized transportation problems, the ua, vg are uniquely 
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determined by (9-47). It is not possible to assign an arbitrary value to 
one or more of these variables. 

When z is to be minimized and the basic solution is not optimal, the 
vector Ps, to enter the basis is computed, as usual, from 2, — cs = 
max (25 — cij), zij — cij > 0. To determine the vector to lenve the basis, 
the y$? must be found. Frequently, this can be done very easily by solving 
directly the set of equations (9-43) which, just as the set of equations for 
the ua, vs, can often be solved sequentially. It is also possible to compute 
the yz, by finding a loop in the tableau connecting cell (s, t) and the 
basic cells. However, the technique here is not so simple as in the case 
of transportation problems because vectors outside the loop may be 
needed, and because one can get off to wrong starts in going around the 
loop. These difficulties will be illustrated in the example of the next 
section. 

"The criterion used in the simplex method determines the vector Por to 
leave the basis, i.e., 


zB 8, a 
vm = 0 = min 8. y > 0. (9-49) 
The new basic feasible solution is most easily found by setting х, = 0, 
Zo, = О and making the appropriate adjustments in the tableau to obtain 
а new solution. This is also illustrated in the example. Of course, the 
new basic variables can be computed from the standard transformation 
formulas 

dis = 2 — буй, 

fuc 0. (9-50) 


The technique discussed in this section can also be used to solve some- 
what more general problems in which (9-40) is replaced by 


m 
Met E Xj = b; jed..,m. (9-51) 
ici 
A variety of the algorithm described here has been programmed for 
the IBM 709 digital computer by K. Eisemann. Hence, generalized 
transportation problems can be solved efficiently on this computer, and 
it is not necessary to use the relatively inefficient. simplex method for 
problems which are too large to be worked out by hand. The generalized 


transportation problem has been discussed by Charnes and Cooper [2], 
and Ferguson and Dantzig [5]. 


9-15 Generalized transportation problem; an example. An example 
should help a great deal to clarify the method (discussed in Section 9-4) 
for solving generalized transportation problems. Consider. the problem 


4 


1 


| 


——————— а 
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given in Table 9-17; assume that we wish to minimize the cost 2 = Усулу. 

There are three resources and four requirements (products), and there- 
fore a nondegenerate basic feasible solution should have exactly seven 
positive x;; To determine the initial basic feasible solution, we shail use 
the column-minima method. The lowest cost in column 1 is с = 0.25, 
and hence we set 


Bici (а) ih (2, aD = 200. 


The first requirement has been satisfied, and we move to column 2. The 
lowest cost is сзә = 0.20; we set 


X32 = min (o. #) = min (100, ud = 400, 
and the second requirement is satisfied. In column 3, соз is the lowest 
cost; we set хоз = 500 and satisfy the third requirement. 

Column 4 presents more of a problem. The lowest cost is co, = 0.40. 
However, we cannot set £24 = 1000 because there are not enough units 
of resource 2 to do this. We have used 0.30(500) = 150 units of resource 2 
in satisfying the requirement of column 3. This then leaves 350 units of 
resource 2. If we devote these 350 units to requirement 4, we have 
224 = 550/0.40 = 875. Now all units of resource 2 have been used. 
However, we have not satisfied requirement 4. The next lowest cost in 
column 4 is спа = 0.50. We have used 0.35(200) = 70 units of resource 1 
in satisfying the first requirement and thus have 130 units left. To satisfy 
requirement 4, we must set x14 = 125 so that 214 + Z24 = 1000. This 
uses 0.50(125) = 62.5 units of resource 1. All the requirements have been 
satisfied. However, 200 — 70 — 62.5 = 67.5 units of resource 1 and 
400 — 0.40(400) = 240 units of resource 3 have not been used. Thus 
we have a solution with slack in resources 1 and 3. The values of the 
slack are entered in the table. We have found a unique feasible solution 
with exactly seven positive туу. Hence we have a nondegenerate basic 
feasible solution. The positive z;; are the circled values in Table 9-17. 

To determine whether the above solution is optimal, we must compute 
the zi; — су. This is done by solving the set of equations (9-47). For 
our case these equations are: 


0.35u, + v, = 0.25, 0.40из + v2 = 0.20, 0.30u2 + оз = 0.30, 
0.40u2 + v4 = 0.40, | 0.50u, +24 = 0.50, ш = ug = 0. 


We immediately find v, = 0.25, v2 = 0.20, vs = 0.50, us 0.25, 
оз = 0.375. The z;; — cj; = dijui + vj — cij can then be computed. 
They are listed in boldface type in Table 9-17. 
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Тави: 9-17 


"There is one positive гу — су, namely гүз = сүд. Thus pia should 
uS the basis. In order to determine the vector to leave the basis, the | 
315 must be found. This is done by solving the system of equations (9-43). 
For this particular case, they become " 


0.3513 + 0.5014 + yis = 0.35, — 0.30/13 + 0.1011 = 0, 
04013 + ib = 0, yid = 0, el = 0, 33 
yis = 1, yis T 1% 


Note that it is very easy to write down these equations. ‘There is an 

equation for each row and each column (except the slack column). For 
example, to obtain the first equation above, we turn to the first row of 

Table 9-17. There are three cells of this row in the basic solution: (1, 1), 

(1, 4), (1, 5). Thus the left-hand side of the equation is йш + ШЕ 4 

Yih; the right-hand side is d; since the first component of pis is dia: 

The solution to these equations can be written down at once; it is 


йз = о уд; ч 
vis=1, 05 = 0, 
у = —075, 014 = 0.75, 
yis = —0.025. 


4r 
Consider what happens if we attempt to evaluate the 7 "8 by means of 
the loop technique used for transportation problems. T m is a unique _ 
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loop connecting pia, Pia, P24, Pa. Thus we might, attempt, to write 


pis = 0.35e1 + е = ae (0.50е; + ez) 


(0.406, + ez) + 0:35(040) (озде, + eu). 


_ 035 03504 
0.50(0.30) 


0.50 


However, this does not, work out, since on the left we have eg and on the 
right (0.35(0.0)/0.50(0.30)]es. Furthermore, there is no other vector in 
the basis which contains eg. Suppose that we traverse the loop in the 
other direction. Then we ean write (note that now eg is in роз) 


pis = 0.35e1 + е = (0.3063 + es) — 030 (0406; + ел) 


0.30 
T 040 (0.50е; + e;) —0.025e,. 
Here we have expressed руз in terms of the basis vectors, and the у%% 
are the same as above. However, to do this, we had to use the slack 
vector ej, which was not in the loop. 


Only 12%, 14 are positive. The variable to leave is then found from. 


Thus z,4 leaves the basis and is replaced by тз. The new basic feasible 
solution can be found most quickly by adjusting the values in the tableau 
to obtain a new solution. ‚We know that 2:3 = 0 = 166.7. Hence to 
have the correct sum in column 3, it must be true that 253 = 333.3. 
Then since £14 = 0, we obtain 224 = 1000. Now all that remains to 
be done is to adjust the slack. In the new solution, we are using 
0.35(200) + 0.35(160.7) = 128.4 units of resource 1. Thus х,у = 200 — 
128.4 = 71.6. For the second resource we should again obtain an exact 
balance. This is indeed true since the amount used of this resource is 


0.30(333.3) + 0.40(1000) = 500, 
and there is zero slack. Everything else remains unchanged. 


The new basic feasible solution is given in Table 9-18. To obtain the 
Zij — су, we first determine the new Ua, vg by solving the set of equations 
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Taste 9-18 


—0.167 | 500 | —0.1607 


(9-47) corresponding to the new solution. These equations are: 


0.35u, + v, = 0.25, 0.35ш + оз = 0.35, ш = 0, 0.30u2 + va = 0.30, 
0.40u2 + v4 = 0.40, 0.40из + ve = 0.20, из = 0. 


The solution is шу = 0, vı = 0.25, оз = 0.35, из = —0.1667, v4 = 0.466, 
из = 0, v; = 0.20. 

The zij — су are then computed from (9-48); they are listed in 
Table 9-18. All 2;; — суу are negative, and consequently, an optimal 
solution has been found. Since this problem can actually be solved by 
inspection, it is clear that we have indeed obtained an optimal solution. 
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PROBLEMS 


9-1. Consider a transportation problem involving 4 origins and 6 destinations. 
Write out explicitly the matrix A of Eq. (9-7). By evaluating the determinant 
of the matrix whose columns are the vectors P21, p22, P12, риз, Рэз, P34, P44, 
pas, p4o show that these vectors are linearly independent and yield a set of basis 
vectors for. the transportation problem. Express P16, Pat in terms of the basis 
vectors, and write out the vectors explicitly. 

9-2. Choose any 8th-order minor |As| from matrix A of Problem 9-1 and show 
that |As| = +1, 0. 

9-3. Let B be the matrix of the basis vectors referred to in Problein 9-1. 
Show that every 9th-order minor of B is different from zero. 

9-4. Solve the transportation problem for which the costs, origin availabilities, 
and destination requirements are given in the following tableau by the stepping- 
stone method and use the northwest-corner rule to obtain an initial feasible 


solution. 


O1 1 2 1 4 5 
02 3 il 2 H 4 
0s 4 2 1 5 9 6 
04 3 1 7 3 ¥ 4 
b; | 20 [М 30 10 ee 
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9-5. Solve the preceding problem by means of the uv-method of computing 
the 2; — ci. 

9-6. Draw the basic tree for each tableau of Problem 9-4. 

9-7. Solve Problem 9-4, using the row-minima technique to find an initial 
solution. Compare the number of iterations required to obtain an optimal solu- 
tion with the number of iterations required in Problem 9-4. 

9-8. Solve Problem 9-4, using the column-minima technique to find an initial 
solution. Compare the number of iterations required with that of Problem 9-4. 

9-9. Solve Problem 9-4, using the matrix-minima technique to find an initial 
solution. 

9-10. Solve Problem 9-4 by means of Vogel’s method for finding an initial 
solution. 

9-11. Solve the problem discussed in Section 9-8, using Table 9-11 for an 
initial solution, А 

9-12. Solve the problem diseussed in Section 9-8, using Table 9-12 for an 
initial solution. 

9-13. Solve the problem discussed in Section 9-8, 
initial solution. 

9-14. Solve the problem diseussed in Section 9-8, 
initial solution. 

9-15. Solve the problem whose initial solution is given in Table 9-8. Carry 
along the e's explicitly. 

9-16. Solve the following transportation problem by the uv-method, using 
the northwest-corner rule. Do not introduce the еѕ explicitly to remove the 
degeneraey problem. Simply keep track of the variables which are at a zero 


using Table 9-13 for an 


using Table 9-14 for an 


level in the basic solution. : 
Dı Рз Da Ds | Ds De ai 
us 
01 5 10 15 8 9 7 30 
O2| 14 13 10 9 20 21 40 


9-17. A company has three warehouses, numbered 1, 2, 3, containing 10,000, 
5,000, and 16,000 units of its product. In the next month, 2,000; 1,000; 3,000; 
4,900; 500; 600; 950 units must be shipped to seven retail outlets numbered 


\ 
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1,...,7. The unit-cost of shipment from any warehouse to any retail outlet 
is contained in the following matrix. Find the minimum-cost shipping schedule. 


| 
9-18. Solve the following transportation problem by the uv-method, using 


any one of the available methods for finding an initial solution. 


9-19. Solve the following transportation problem by the stepping-stone 
method; use any technique desired for obtaining an initial solution. 


Di D2 Рз Da a; 
O1 2.67 —1.05 1.30 —2.00 45 
02 1.45 —0.50 0.40 —0.05 30 
Оз 3.00 2.00 1.50 1.40 30 
b; 25 15 40 25 105 
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9-20. Solve the following transportation problem by the uv-method, using 
any one of the techniques for obtaining an initial solution. 


9-21. Show that if R contains as columns the т 4# п — 1 basis vectors 
рё, then the set of equations (9-28) сап be written (u, VR = cy. Next show 
that if we set to zero one component of (u, v), say component k, and if T is the 
matrix formed from R by crossing out row k, then the remaining m + n — 1 
components of (u, v) are given by cT 71. Reconcile the tact that one of the 
components of (u, v) is arbitrary with the theory of the simplex method, where 
cB”! is uniquely determined. Hint: Let B be the corresponding basis matrix 
of order m + n if the transportation problem was solved by the simplex method. 
Then B can be imagined to contain R in its first т + n — 1 columns and an 
artificial vector in the last column. Note that any artificial vector can be in 
the last column. Imagine that the price of the artificial vector is zero. 

9-22. How is an optimal solution to a transportation problem changed if a 
constant À is added to each cost in one row i of the transportation tableau? 
By how much is the optimal value of the objective function changed? 

9-23. How is an optimal solution to a transportation problem changed if a 
constant À is added to each cost in one column j of the transportation tableau? 

` By how much is the optimal value of the objective function changed? 

9-24. Consider the transportation problem 


^ 
Уа X a; $21,...,m, 


j=l 


Fue уі, 


i=l 


minz = bJ egt 
ij 


for шуу > 0. Assume that all с; > 0 and Da; > Eb; Prove that, in any 
optimal solution, the constraints >; ту > b; will hold as strict equalities, i.e., 
no more than necessary will be shipped. 
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9-25. Suppose that we are maximizing z in Problem 9-24. Prove that, in 
any optimal solution, the constraints У; zi < a; will hold as strict equalities 
What is the physical interpretation of this result? 

9-26. Consider the following transportation-type problem: 


a 
ушу >а, өен т; 
ј=1 


ч 
Ple Eb -L.gh5 


i=l 


minz = PX сй. 
Ыы 


For cij > 0 prove that, in an optimal solution, each of the first m constraints 
will hold as'a strict equality if У; > 22a; and each of the last n constraints 
will hold as a strict equality if a; > by. 

9-27. Consider the transportation-type problem 


A 
Vrs = a, i= 1,...,m, 


j=l 


Dory =b, jl..." 
i=l 
maxz = 25 Cifti. 
a 
for z;; > 0. Convert the constraints to a set of equations and introduce a new 
redundant equation so that Za; = $ b;. Write out the matrix of the coefficients. 


Show that this matrix is unimodular. 
9-28. How must the stepping-stone algorithm be modified to solve Problem 


9-27? How can we obtain an initial basic feasible solution? 

9-29. How must the uv-method be modified to become appliable to Prob- 
lem 9-27? 

9-30. Consider the transportation-type problem 


Д 
о СЯ 


j=l 


т 
YES) jahat 
il 
minz — У сз 
ay 
for 2:; > 0. The c;; may have any sign. Convert the constraints to a set of 


equations and introduce two additional equations so that Уа; = $b; Write 
out the matrix of the coefficients. Show that this matrix is unimodular. 
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9-31. How must the stepping-stone algorithm be modified to become ap- 
plicable to Problem 9-30? 

9-32. How must the ur-method be modified to treat Problem 9- 30? 

9-33. Consider the usual formulation of the transportation problem, that is, 
Eqs. 9-1 through 9-4. Suppose that certain z;; are required to be zero. Show 
how to apply the stepping-stone or uv-methods to such a problem. Hint: Can 
one use а — M-trick here? 

9-34. Show how to solve the linear programming problem 


29399 $a, т 
j2i 


у ушу = bd 


i<j 
minz = > Citij 
(Si 


1.218 


for non-negative tij. 

9-35. Show how to incorporate artificial variables within the tableau format 
of Table 9-16 if they are needed to obtain an initial basic feasible solution to & 
generalized transportation problem. How are the zi; — су computed when 
artificial vectors are in the basis? 

9-36. How does degeneracy appear in the process of determining an initial 
basic feasible solution to a generalized transportation problem by means of one 
of the methods used to obtain basic solutions to transportation problems? Find 
a simple way of adding cells at a zero level to yield a degenerate basic feasible 
solution. What problems arise if one wishes to use a perturbation scheme which 
rigorously eliminates the possibility of degeneracy? 

9-87. Solve the following generalized transportation problem: The upper 
number in each cell is the cost cij, and the lower number is dij. Assume that 
the objective function is to be minimized. 


Pi P2 P3 P4 Ps Slack а; 


2.4 1.5 Joke 0.95 | 3.0 
Ri : 200 
3.2 2.3 44 1.5 3.5 
0.75 | 0.90 | 1.2 | 1.0 1.2 
Re! 12 1.4 1.5 1.4 1.7 
3.5 1.4 Pa epi 4.0 
Rs) 49 2.0 2:5 |-:318 4.5 
b; | 50 40 80 150 210 
LANNE 
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9-38. Under certain circumstances the techniques used to find initial basic 
feasible solutions to transportation problems will not work for generalized 
transportation problems. Suppose, for example, that the d;; are inversely 
proportional to the ¢;;. Thus the selection of a solution with the lowest ci; may 
require more resources than are available, beeause the d;; are largest for the 
small сг. However, there may be a feasible solution if activities with higher 
costs are used. How can one obtain an initial basic feasible solution in this case? 

9-39. In Section 9-14, we presented an exaet procedure for solving gen- 
eralized transportation problems. Sometimes such problems can be solved 
approximately by scaling them so that all the coefficients are close to 1. Then 
we set all coefficients equal to 1 and use the transportation method as an approx- 
imation, The scaling is carried out by multiplying the constraints by a constant 
(which does not change the constraint) and by multiplying the activity vectors 
by some constant. (How does this affect the corresponding variable?) Discuss 
in detail how this scaling technique works, and indicate the conditions under 
which a generalized transportation problem can be reduced to a problem whose 
nonzero coefficients are all close to 1. Try to use the scaling technique on the 
example of Section 9-15. 

*9-40. Discuss the way in which a digital computer might be used to solve a 
transportation problem by the uv-technique. Note that to determine the unique 
loop connecting the basis cells and the cell to enter the basis, the computer must 
be provided with a definite set of instructions for finding the loop. Develop such 
a set of instructions. Construct a flow chart for solving a transportation problem 
by the uv-method on a digital computer. 

*9-41. Discuss the problems involved in developing a code for solving the 
generalized transportation problem on a digital computer. 

9-42. For the tableau of Problem 9-4 assume that the origin availabilities 
are 50, 75, 75, 100, respectively, instead of the values shown. Add a slack 
column and solve the problem using the uv-method. 

9-43. For the tableau of Problem 9-16 assume that the origin.availabilities 
are 45, 70, 30, 100, respectively, instead of the values shown. Add a slack 
column and solve the problem using the ue-method. 

9-44. For the tableau of Problem 9-18 assume that the origin availabilities 
are 60, 50, 120, 70, 60, respectively, instead of the values shown. Add a slack 
column and solve the problem using the uv-method. 

9-45. For the tableau of Problem 9-4 assume that the b;-values are 10, 15, 
20, 10, 30, 20, respectively, instead of the values shown. Assume that the re- 
sulting tableau provides the data for a problem of the form given by Eq. (9-35). 
Add a surplus row to the tableau and solve the problem, using the w-method. 

9-46. For the tableau of Problem 9-18 assume that the b;-values are 20, 10, 
30, 50, 40, 15, 10, respectively, instead of the values shown. Assume that the 
resulting tableau provides the data for a problem of the form given by Eq. (9-35). 
Add a surplus row to the tableau and solve the problem using the uv-method. 


* Starred problems assume that the reader has some familiarity with digital 
computers. 
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9-47. For the tableau given in Problem 9-18 assume that the a;-values are 
70, 50, 100, 50, 90, respectively, instead of the values shown. Assume that the 
resulting tableau provides data for a problem of the form given by Eq. (9-38), 
where the objective function is to be minimized. Add a slack column and a 
surplus row and solve the problem by the uv-method. Verify the results of 
Problem 9-24. 

9-48. For the tableau given in Problem 9-19 assume that the a;-values are 
75, 40, 60, respectively, instead of the values shown. Assume that the resulting 
tableau provides data for a problem of the form given by Eq. (9-38), where the 
objective function is to be minimized. Add a slack column and a surplus row 
and solve the problem by the we-method. Do the results of Problem 9-24 hold 
here? 

9-49. A square matrix B is said to be triangular if bi; = 0 when i > j, that is, 
all elements below the main diagonal are zero. A system of n equations in п 
unknowns Bx — b is very easy to solve if B is nonsingular and triangular, for 
we have immediately z, = b,/b,,. Then z,..; can be found-after substituting 
in the value of £n, etc. The variables can be determined sequentially. For each 
of the tableaux given in Tables 9-2 through 9-6 write down the set of m - n 
equations in m +- n — 1 unknowns which determine the basic variables. After 
dropping any one of the redundant equations in each set, show that the resulting 
system of m + n — 1 equation in m+ n — 1 unknowns can be arranged so 
that the matrix of the coefficients is triangular. For each of the tables referred 
to above, write down the set of equations which determine the u; vj. After 
setting the value of any one of the u; or v; arbitrarily in each set of equations, 
show that the resulting system of m + n — 1 equation in m + n — 1 unknowns 
can be arranged so that the matrix of the coefficients is triangular. 

9-50. Prove in general that for any transportation problem, the set of 
m+ n — 1 equations in m+ n — 1 unknowns which determine a basic solu- 
tion can be arranged so that the matrix of the coefficients is triangular (see 
Problem 9-49). Prove the same result for the set of equations which determine 
the u; vj. Hint: This requires that we show that one variable can be determined 
immediately and that this is also true for another variable in the resulting sys- 
tem of equations, after the value of the variable originally determined has been 
substituted back in. Use the type of analysis given on p. 302. 


CHAPTER 10 
NETWORK FLOWS 


“The thought beneath so slight a film 
Ts more distinctly seen,— 
As laces just reveal the surge, 
Or mists the Apennine.” 


Emily Dickinson. 


10-1 Theory of graphs. In Section 9-5 we introduced (without ex- 
plicitly saying so) some material from the theory of graphs. In the last 
few decades, this sub)2ct has assumed an important place in the study 
of electrical networks and has even found its way into introductory 
texts on electrical engineering [8]. More recently, the theory of graphs 
has become of interest in economies and other fields, in connection with 
transportation networks and other flow problems. The computation of 
maximal flows in networks and the usefulness of this theory in solving 
linear programming problems of the transportation type will be the 
subject of this chapter. In order to introduce some terminology and 
definitions which will be useful in the development of this material, we 
shall first very briefly discuss some elementary topics in the theory of 
graphs. 


GRAPH: Consider а set of two or more different "points," with certain pairs 
of these points joined by one or more "lines." The resulting form is called 
га graph, and can be denoted by G. 


The “points” referred to in the above definition will be called nodes, or 
vertices, of the graph. What we here call nodes were the cells of the 
tableau in Section 9-5. A “line” joining two different nodes will be called 
a branch, or edge, of the graph. The nodes of the graph will be denoted 
by numbers, i.e., 1, 2,... We shall discuss only problems whose graphs 
contain a finite number of nodes. Any branch of a graph can be char- 
acterized by the pair of nodes which it joins. More than one branch 
may connect any two nodes. The two nodes joined by a branch are 
referred to as the end points of the branch. The degree of a node 2 is the 
number of branches of the graph which have node 7 as an end point. 
A node of degree 1 is called an extreme point of the graph. A branch one 
of whose end points has degree 1 is called a terminal branch. 

Usually, the nodes of a graph are considered to be points in euclidean 
space, and the branches are straight lines or curves joining these points. 
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From the abstract point of view, however, a node can be any element 
whatever. A branch has only one characteristic: It defines two end 
points. Thus from the purely abstract point of view, the theory of graphs 
is the theory of combinatorial analysis. We shall always think of a graph 
as a set of nodes and branches which can be represented by some con- 
figuration drawn in a plane (for example, Fig. 10-1). 

We can attribute a sense of direction to a branch by stating which 
node is to be considered the point of origin. Such a branch will be called 
oriented. When drawing a graph, we indicate the orientation of a branch 
by an arrowhead. An oriented branch with node 7 as the point of origin 
and node j as the point of termination can be represented algebraically 
by (i,j). If an orientation is assigned to each branch of a graph, the 
resulting graph is said to be oriented. If there are k branches in the graph, 
then it is possible to form 2* oriented graphs. We shall use (i, j) to repre- 
sent a nonoriented branch joining nodes 1, J. 


Pari: A path joining nodes 1, j is an ordered set of branches, 


(i p), (p,a), (a 0), ..., (6м), (uj), 


such that each node in the ordered sel, with the possible exception of the 
first and last nodes, is the end point for two and only two branches in the 
set. The nodes i, j are called the extremity points of the path. 


ExAMPLE: The following sets of branches form two different paths join- 
ing nodes 1, 4 in Fig. 10-1. 
(à) (1,2), (2,3), (3,6), (6, 4), 
(b) (1,2), 2,3), (3,5), (5,4). 
Intuitively, a path is à continuous broken line joining the extremity 
points. An oriented path is obtained from the path defined above if the 


branches are oriented such that the terminal point of one branch is the 
point of origin for the immediately sueceeding branch. 


Loop: If a path is defined so that i = J, i.e., the extremity points of the 
path are one and the same node, then the path is called a loop. 
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CONNECTED GRAPH: A graph G is said to be connected if there is a path 
in © joining any two nodes of the graph. 


If we remove from a connected graph G one of its extreme points and 
the corresponding terminal branch, the resulting graph G' is also con- 
nected.. To prove this we note that any path in G which involves the ex- 
treme point removed must have the extreme point as one of its extremity 
points. Hence any path in G connecting two nodes of G’ cannot involve 
the removed extreme point. Thus any path of G connecting two nodes 
of G' is also a path of G', and hence G' is connected. 


Tree: A tree is a connected graph which has no loops. 


A connected graph is a tree if and only if the path joining any two 
nodes is unique. If every path is unique, then there is no loop, for if there 
were a loop, 


(i, р), (р, Ф, (or) -> -> (6 u), (2), 


there would be two distinct paths, (i, p), (p, д) and (q, r), . . .., (t, u), (u, 3) 
joining the nodes 7, q. On the other hand, if the graph is a tree, a path 
must be unique in every case, for if there were two distinct paths joining 
i, j, they could be combined to yield a loop. 

It is easy to show that a tree always has at least two extreme points. 
To prove this choose any node т. If i is an extreme point, it is only 
necessary to show that there is at least one more extreme point. There 
is a branch joining 7 to another node, say j. Either j is an extreme point 
or there is a branch joining j to another node k, etc. Since we have only 
а finite number of nodes and no loops, this process must terminate at an 
extreme point different from 7. If 7 is not an extreme point, then at least 
two branches lead from 4 to different nodes, say j and k. We can then 
follow along paths from j and k, and each of these must again terminate 
in an extreme point. Furthermore, the paths have no elements in common 
or a loop could be formed. Hence the two extreme points are different. 

If a tree has N nodes, it has precisely N — i branches. This can be 
proved by induction. When N = 2 and there is no loop, then there can 
be only one branch joining the two nodes. Hence the result holds in 
this ease. Suppose that the result holds for N — n. Consider a tree with 
n + 1 nodes. Let the number of branches be k. Locate an extreme point 
of this tree and remove this extreme point and the corresponding terminal 
edge. By the above proof, the resulting graph is connected and has no 
loops. Therefore it is a tree with n nodes. By hypothesis, it has n — 1 
branches, Thus k = n — 1+ 1 = n, and the theorem holds for N = 
n + 1 if it holds for N = n. Hence, by induction, it holds for all integers 
М> 2. 
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These simple theorems should give the reader some feeling for the 
manner in which propositions in the theory of graphs can be proved. 

It is often useful to imagine that some sort of flow (electric current, 
some fluid, goods, money, etc.) occurs in the branches of a graph, 


Network: A network is a graph such that a flow can take place in the 
branches of the graph. 


A network may or may not be oriented. The orientation of any branch 
is now taken to be the direction of flow. If there is a limit to the magni- 
tude of the flow in any branch of a network, then a capacity restriction is 
imposed on that branch. It is possible to have capacity restrictions on 
the nodes as well as on the branches. A node j is called a source if every 
branch which has this node as an end point is oriented in such a way that 
the flow in the branch moves away from j to another node. A node i 
is called a sink if every branch which has this node as an end point is 
oriented in such a way that the flow is from other nodes to т. 

In a network, we assume that the branches are connected only at 
nodes. In Fig. 10-1 for example, the branches (4, 6), (3, 5) eross; however, 
it is assumed, that they are not interconnected, i.e., a flow in (4, 6) cannot 
enter (3, 5) at this point of intersection. In this particular case, we could 
have drawn the figure so that the branches would not intersect. When 
a network is drawn in a plane, we place no restrictions on the way its 
branches may cross one another. The understanding is, however, that 
they are connected only at nodes.. In electrical engineering, it is sometimes 
necessary to be more specific and to assume that the network can be 
drawn so that the branches do not cross one another; i.e., it must be 
assumed that the network is topologically planar. 

In the next section, we shall discuss the problem of finding maximal 
flows in connected networks containing one source and one sink. These 
results will be useful in the development of another algorithm for solving 
the transportation problem, which is even more efficient than that dis- 
cussed in the preceding chapter. 


10-2 Maximal flows in networks; intuitive approach. Consider a con- 
nected network consisting of a single source, a single sink, and some 
intermediate nodes. We shall assume that the network is oriented, and 
that there is a capacity restriction d; > 0 (whieh may be infinite) on 
each branch (7,7). It will be assumed that there are no capacity restric- 
tions on the nodes. All other cases of interest can be reduced to this 
case, and in some of the problems the reader will be asked to show how 
this can be done. A typical network is shown in Fig. 10-2. We wish to 
determine a procedure for computing the maximum possible flow from 
source to sink. 
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FicunE 10-2 


The flow in the networks to be considered may be flow of fluids, elec- 
tricity, funds, automobiles, ete. The physical units of the flow may be 
cubic feet/sec, amperes, dollars per month, ete. In general, the flow 
will be a rate. We shall be interested in steady-state flows, i.e., flows 
which, in any branch, do not change with time. It is assumed that, the 
flow in each branch is infinitely divisible, and that for a branch Gj J) it 
can have any value <j; satisfying 0 < z;; < dij. Furthermore, if there 
is a flow z;; in branch Gi 3), then the flow in the branch can be increased 
by any amount up to а maximum of g;; = dij — туу. It is also assumed 
that there is conservation of flow at any node other than the source and 
sink, That is, the flow into the node is equal to the flow out of the node. 

Thus, i in Fig. 10-2, if the flow through (8, | (8, 8, 10) i is 2 and the flow through 
(7, 7, 10) is 1, then the flow in branch (10, 13 18) is 2+1 = 3. 

А flow іп а network is described by a set of non-negative nu numbers Tij 
which give the flow from node 7 to node j through branch Gj 2). A flow 
is said to be feasible if there is conservation of flow at every node except the 
source and sink, and if 0 < zi; < dj; for all i,j. Any feasible flow deter- 
mines a quantity z which is flowing from the source to the sink. A feasible 
flow in a network is said to be maximal if the amount flowing from source 
to sink is finite, and if there is no other feasible flow which yields a larger 
flow from source to sink. If there is no finite upper bound on the flow 
in some of the branches, it may turn out that the flow from source to sink 
can be made arbitrarily large. 

Consider a given network and let any feasible flow be denoted by a 
set of туу. If the flow is feasible, it must satisfy the capacity restrictions 
and the conservation of flow at the nodes other than the source and 
sink. If there are N nodes in the network, we number the nodes so that 
1 is the source and № the sink. In order to have conservation of flow at 
the nodes, it must be true that 


eb. ea (10-1) 
k T 


the sum being taken over all k for which there can be flow into i, and 
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all r for which there can be flow out of i. These N — 2 constraints, 
together with the capacity constraints 
OS 25: < аттар, (10-2) 


represent the restrictions on the problem. In order to find the maximal 
flow, we must maximize the flow z from the source to the sink; i.e., 


z= tw (10-3) 


the sum being taken over all branches leading from the source, subject 
to the constraints (10-1) and (10-2). Since we have assumed that no 
accumulation occurs, z must also be given by 


zc > TaN, (10-4) 
q 


the sum being taken over all branches leading to the sink. It follows that 
the problem of finding the maximal flow in a network is reducible to à 
linear programming problem. Hence, in theory, we have solved the 
problem. However, it is possible to develop a much more efficient compu- 
tational procedure for finding maximal flows in networks. Я 

Given a diagram of а network such as Fig. 10-2, and assuming that 
the capacity of each branch is labeled, intuition suggests the following 
technique for finding a maximal flow in the network. We begin at the. 
source and move along branches of positive capacity until we determine 
a path from the source to the sink. We search this path for the branch 
with the smallest capacity d and set the flow in this path equal to d. 
Then we decrease by d the capacity of each branch in the path and con- 
sider the resulting network. The capacity of any branch in the above 
path whose original capacity was d is now zero. We repeat the same opera- 
tion for the new network, and continue until there is no path from the 
source to the sink such that each branch in the path has positive capacity. 
The maximal flow is then the sum of the flows in the paths which were 
obtained at each step. 

Our intuitive idea is sound and will indeed yield a maximal flow, pro- 
vided that we clear up one point. It is quite possible that we can find 
paths from the source to the sink such that one or more of the branches 
have a wrong orientation; i.e., a flow from the source to the sink would 
pass through these branches in the wrong direction. However, it may 
turn out that the flow in the network can be increased if we imagine that 


a flow imposed in the wrong direction decreases the flow in the proper | 
direction. Our intuitive idea will work, provided we allow for the possi- | 
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bility that à flow may be imposed in the wrong direction on some branch 
if, for that branch, there is a flow in the proper direction. However, we 
never allow a net flow in the wrong direction. 

The capacity dj; for a flow in branch (7,7) will be assumed to refer to 
the net flow. We imagine that two flows, . xi and zi, pass simultaneously 
in opposite directions through branch Gj DU the net flow is then 


х= tij ху > 0, (10-5) 


and must satisfy (10-2). The net flow is the only quantity of physical 
interest, but for mathematical reasons, we must allow for the fictitious 
flows in opposite directions. - Given a value of ti; then 27; must lie in 
the interval 0 < 2$; X xi. This means that we cannot impose a flow 
in the wrong direction along a branch unless we have at least as great 
a flow in the proper direction. There cannot be a net flow in the wrong 
direction. If we are given a value of xi, then x}; must satisfy xus 
zi S di + 2j 

There is no reason that there cannot be a provision for an actual phys- 
ical flow in either direction between the nodes т, j of a network. On a 
diagram, this possibility is | usually indicated by two directed branches 
(5,2), (9,2) [the branches (7,8 8), (8,7 7) in Fig. 10-2). In this situation, 
dij and dj; need not have the same value. When there can be an actual 
physical flow in both directions between node 7, j, there exists a loop 
involving only these two nodes. We can note that a maximal flow never 
requires that both 2,;, ху; be positive. To see this, let d = min (туу, ху) 
for any flow in the network. Consider the new variables луу — d, zj — d. 
One of these new variables will be zero. This change in variables does 
not in any way alter the flow from the source to the sink. All we did 
was to subtract the loop flow. 


10-3. Maximal flows in networks; examples of the intuitive approach. 
(1) Using the intuitive approach suggested above, let us determine the 
maximal flow in the network shown in Fig. 10-3. The capacities of the 
branches are shown, the node numbers are circled. We begin by finding 
a path from source to sink, i.e., from node 1 to node 6, in which each 
branch has a positive capacity. The path (1, 2), (2, 4), (4, 6) is satisfac- 
tory. The minimum capacity among the branches is 4 for branch (2, 4). 
We thus impose a flow of су = 4 along this path and decrease by 4 the 
capacity of each branch in the path. This yields the network in Fig. 10-4. 

Let us next choose the path (1, 3), (3,4 ‚ 4), (4,6 6). The minimum capacity 
among the branches is 4 on branch (4,6). Thus we impose a flow of 
тз = 4 along this path and reduce by 4 the capacity of each branch. 
This yields Fig. 10-5. 
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All the branches of the path (1, 3), (3,5), (5, 6) have positive capacity. 
The minimum capacity is 3 for branch (3, 5). Acflow of тз = 3 is imposed 
on this path, and all the capacities are reduced by 3. This yields Fig. 10-6. 

Every branch of the path (1, 2), @, 5), (5, 6) has a positive capacity. 
The minimum capacity occurs for branch (2,8) and is 1. A flow of 
94 = 1 is imposed along this branch, and the capacity of each branch 
in the path is reduced by 1. This gives Fig. 10-7. 
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There is no path left along which we can impose a positive flow. For 
example, the path (1, 3), (8, 4), (5,4), (5,6) is not suitable; we cannot 
set zi, to a positive value because 25, = 0. Hence we must have a 
maximal flow, and it is given by с +02 +03 Бо = 4-43 + 
1 = 12. The flow in any branch (7, 3) is found by subtracting the ca- 
pacity of that branch in Fig. 10-7 from the corresponding capacity in 
Fig. 10-3. Thus, for example, тү; = 5, түз = 7. The sum of tiz and 
т\з should be the total flow, and indeed, it is. The reader might note that 
the set of branch flows yielding the maximal flow is not unique. For 
example, we can inerease by 1 the flow along the path (1,3), (3, 4), (4, 6) 
and decrease by 1 the flow in the path (1, 2), (2, 4), (4, 6), and the flow 
from the source to the sink is unchanged. 


(2) Consider the network shown in Fig. 10-8. To find a maximal 
flow let us choose the path (1, 3, (3, 4), (4, 6). The minimum capacity 
is 5; setting a flow of су = 5 in this path, we obtain the new capacity 
diagram of Fig. 10-9. 

'There is now no path from source to sink such that each branch has 
positive capacity, and the orientation of the branch is the same as the 
direction in which the path is traced. However, we do not have a maximal 
flow. Consider the path (1, 2), (2, 4), (3, 4), (3,5), (5, б). Any flow in 
this path moves in the wrong direction along branch (3, 4). However, 
since z34 = 5, we can impose a flow x4, of as much as 5 in the wrong 
direction. The minimum capacity for the path is then 4. If a flow of 
сз = 4 is imposed along this path, there remain the capacities shown in 
Fig. 10-10. The net flow in (3,4) is 254 — 243 = 9 — 4 — 1; hence 
the remaining capacity is 4. 

Clearly the flow is now maximal and has the value с + сз = 9. We 
note that 212 = 4, 213 = 5, T24 = 4, 234 = 1, Т6 = 5, 1735 = 4, 
хув = 4. This illustrates a case where it is necessary to use the mathe- 
matical fiction of imposing a flow in the wrong direction. Note that the 
only result of imposing a fictitious flow in the wrong direction along (3,4) 
was that four units of flow from (1, 3) were routed through (3, 5) rather 
than through (3, 4). The actual flow corresponding to the four units of 
flow in the wrong direction through (3, 4) really went from (2,4) to (4, 6) 
to m make up the four units in (4, 6) which were originally coming through 
(3, 4). 

Since, for complicated networks, the above procedure for finding the 
maximal flow is rather inconvenient, it is preferable to use a straight- 
forward algebraic technique. Such a method can easily be developed 
from the above intuitive notions. The algebraic procedure can then be 
used to prove that the intuitive approach does indeed yield a maximal 
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flow. In a complicated network, it is difficult to keep track of the branches 
which form the path from source to sink. Another time-consuming task 
is the search along the path for the branch with the smallest capacity. 
Both of these difficulties can be eliminated if a labeling process is used 
as we move from one node to the next. The procedure can be system- 
atized in many ways. We shall present the method suggested by Ford 
and Fulkerson [4]. 


10-4 The labeling technique. Imagine that we have a network in which 
a flow may or may not exist. Assume that the capacity of each branch is 
known and that we have labeled the excess capacities of the branches, The 
excess capacities g;; are defined in the following way. When a net flow 
is allowed from 7 to j but not from j to i, write ру = di; — zi; > 0. 
This means that an additional flow of g;; can be imposed through branch 
Gj) from i to j. Furthermore, it is possible to impose a fictitious flow, 
up to туу, from j to i in a direction opposite to the orientation of the 
branch. Thus, write gj; = туу > 0. When an actual physical flow is 
allowed from т to j and j to i, write фу = dij — zij + tji бу = dji — 
Tj; + Tiz 80 that gij, gj; represent the combined excess capacities of the 
two branches in the respective directions. 

To determine whether the flow can be increased, we begin at the source 
and consider all nodes which are joined to the source by branches of 
positive excess capacity. It will be supposed that the source is numbered 1. 
Let the index j refer to the nodes which are joined to the source by branches 
of positive excess capacity. On the diagram of the network, we label node j. 
with two numbers (бу, Yj), where 6; = gi; and Y; = 1. The value of бу 
is the excess capacity from the source to node j. The value of Y j indicates 
the node from which we came to label j; in our case, this node is the 
source. If in doing this we labeled the sink N, so that there is a branch 
of positive excess capacity from the source to the sink, we move to the 
final step (given below), which shows how to increase, the flow. 

Let us take up the case where the sink is not labeled at the first stage. 
For the set of nodes labeled at the first step, we choose the smallest index j. 
Then we investigate whether there are any unlabeled nodes which are 
joined to j by branches of positive excess capacity. If there are no such 
nodes, we move on to the next lowest index j and-repeat the process. 
If some unlabeled nodes can be reached, then, using the general index Ё 
for these nodes, we label each as follows: 


ôk = min (фу, à), — Ye = j. (10-6) 


During this labeling process, we are not concerned about whether the 
orientation of the branch is in the direction of our move from j to k- 
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We only require that the branch have a positive excess capacity (with 
the conventions adopted above for a flow in the wrong direction) for flow 
from j to k. When two branches connect k, j so that a physical flow is 
allowed in either direction, we consider both branches simultaneously, 
that is, we consider the combined excess capacity of the two branches. 
The label 6, on node k gives the minimum excess capacity of the two 
branches which form the path from the source to j to Е. The label Ур 
indicates the node from which we came to label k. 

After having labeled all possible nodes k with the lowest index j, then, 
if the sink was not one of the nodes labeled k, we move on to the next 
lowest index and repeat the process. If the sink has been labeled, we 
jump to the final step given below. This process is continued until we 
have labeled all nodes that are connected by branches of positive capacity 
to the nodes j, or until we have labeled the sink. 

At the third stage, we select from the set of nodes denoted by the 
one with the lowest index. We then look for unlabeled nodes that are 
joined to this node by branches of positive excess capacity. Let these 
nodes be denoted by the general index q. We label nodes q in the same 
way as before, i.e., 


à, = min (gka ёк), fg mis (10-7) 


This process is repeated until, in a finite number of steps, we reach 
one of the two following states: 

(1) No additional nodes can be labeled, and the sink is not labeled. 

(2) The sink is labeled. 
If we reach case 1, the existing flow is maximal. If we reach case 2, the 
existing flow can be increased. ‘Our labeling procedure shows that бу, 
the label on the sink, is the amount by which the flow can be increased 
over the path we have followed in moving from the source to the sink. 
Since the second label on the nodes indicates the preceding node, it is 
easy to work backwards and trace the path. 

Let us denote by gas the excess capacities of the branches in the path 
which led to labeling the sink (in the direction in which we move in going 
from source to sink). Then we consider the new network whose excess 
capacities are 


fag = gos — ÔN, ба = Ова + бу, (10-8) 
фу = gij for branches not in the above path. (10-9) 
Note that when the excess capacity in the direction from o to 8 is de- 


creased, then that in the direction from 8 to œ must be increased by the 
same amount. The entire labeling process is repeated for the- new network. 
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After a finite number of steps, we must reach ease 1, since ôy is always 
strictly positive.* Then we can prove (Section 10-8) that the existing 
flow is optimal. The net flow in each (i, j)-branch is given by 


T = di — gij = gji (10-10) 


when dj; = 0, that is, when no net flow is allowed from j to i. When 
dij dj; are both positive and d;; — g;; > 0, set 


ij dij дару Xj = 0; (10-11) 


if dj; — gj; > 0, set 


їн = dj — 93 Zi... (10-12) 


Note that dj; — gij dj; — gji cannot both be positive since dij — фу = 
Tij — T 

The technique for finding a path from source to sink which results 
from the labeling process described here is somewhat different from that 
suggested in the previous section: we move out from the source and 
consider all possible paths at each step until the sink is labeled. In the 
previous section, we followed a single path as far as possible before back- 
tracking to move out along another path. Both methods will yield a path 
with all branches having positive capacities, if such a path exists. The 
reader should also note that there is no inviolable rule requiring that one 
begin with the labeled node of lowest index at each stage. One could 
begin with any node. However, to avoid mistakes, it is desirable to follow 
a systematic procedure. 


10-5 An example of the labeling process. The labeling process will be 
illustrated for the network shown in Fig. 10-11. The excess capacities 
are labeled on the branches, 

At the first step, we find nodes joined to the source by branches of 
positive capacity, i.e., gı; > 0. These are the nodes, 2, 3, 5. We label 
these nodes with the ordered pair of numbers (êj, Yj), where êj = 01} 
and 7; = 1 since we came from the source. Thus node 2 is labeled (1, 1). 
The first stage is now completed, and the sink has not been labeled. 
Next we select from the set just labeled the node with the lowest index, 
ie., node 2. We find the nodes which are joined to node 2 by branches 
of positive excess capacity, i.e., 3, 7, 10. We ignore node 3, which has 


* This assumes that a maximal flow exists. If the flow can be made arbitrarily 
large, then at some step, dy = ©, and case 1 is never reached. 
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Figure 10-11 


already been labeled. We then label nodes 7, 10. For example, for node 10, 
бо = min (42, ga, 10) = min (1, 2) = 1, 


and Yio = 2 since we came from node 2. 

The sink is still unlabeled; hence we move on to the node with the next 
lowest index among those labeled at the first stage. This is node 3. No 
unlabeled nodes are joined to node 3 by branches of positive excess ca- 
pacity, and we move on to node 5. Here we can label node 9, At this point, 
we have exhausted all nodes labeled at the first stage. During the second 
stage we labeled nodes 7, 9, 10. The sink remains unlabeled. 

Beginning with the node of lowest index labeled at the second stage, 
i.e., node 7, we look for unlabeled nodes connected to node 7 by branches 
of positive excess capacity. We see that node 11 can be labeled. Moving 
out from node 9, we do not find any node to be labeled; moving from 
node 10, we can label node 12. The nodes labeled at stage two have been 
exhausted, and in stage three we have labeled nodes 11, 12. The sink was 
not labeled. 

The node of lowest index labeled in stage three is 11, Now the sink is 
joined to node 11 by a branch of positive excess capacity. At this point, 
the labeling process stops. The flow in the network can be increased. 
The value of буз indicates that it can be increased by one unit along the 
path traced out in the move from source to sink. We can easily work 
backwards to find the path. The label on, the sink shows that we came 
from node 11. From node 11 it is seen that we came from node 7. The 
label ¥; on node 7 shows that we came from node 2 which, of course, was 
preceded by the source. The path is (1, 2), (2,7), (7, 11), (11,13). To 
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find the maximal flow in the network, we would repeat the whole process 
for the network having the excess capacities 


411—915: —1—1—0, a-—95--55—3-1-—4 
зт =1—1=0, 472 = 2, fmu = 1-1 0; 
бфил=4+1=5, fmi = 1-1 0, 


$1511 = 5 + 1 = 6, $5 = gü for all other ї, j. 
A maximal flow is achieved when the sink cannot be labeled. 


10-6 The matrix solution. We can completely characterize any network 
without presenting a diagram. For a network of N nodes, we only need 
the Nth-order matrix of the capacities ||dij. From this matrix it is pos 
sible to draw a diagram of the network because the d;; indicate which 
nodes are joined by branches and what the orientation of the branches is. 
When d;; > 0, there is a branch from i to j which allows a net flow from 
itoj. If di; = 0, no net flow is allowed from 7 to j, and hence, as far as 
flow is concerned, there is no branch (7,3). Of course, for a given network 
diagram, we can obtain different matrices 19:3, depending on how the 
nodes are numbered. The numbers appearing in these matrices will 
always be the same, but they can be changed around by numbering the 
nodes in different ways. 

One can then compute the maximal flow in a network without ever 
using a diagram. The solution can be reached by constructing a series 
of excess capacity matrices ||g;;].. It is easy to perform the labeling process 
since only а |g;;|l-matrix and two additional columns giving êj, y; have 
to be used. The appropriate tableau format has the form shown in 
Table 10-1. 

Given the excess capacities, the labels 35, Yj are easily determined by 
means of the tableau form of Table 10-1. We begin in row 1 of the tableau 
and look for columns having 91; > 0. For such columns, we set à; = gij 
Y; = 1. Assuming that a value has not been assigned to dy at the first 
stage, we move on to the row of lowest index, say i, that was labeled 
at the first step (row 2, perhaps). In row ? we look for gj; > 0 for which 
TOW j has not been labeled. "Then we set à; = min (giz, 8), Yj = È 
If no value was assigned to бу, we move on to the row (labeled at the 
first stage) with the second lowest index and repeat the procedure. We 
continue until a value js assigned to ôy, or until all the rows labeled at 
the first stage have been exhausted. 

If no value has been assigned to бу, and all rows labeled at the first 
Stage are exhausted, we move on to the rows which received a label at 
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TABLE 10-1 


TABLEAU Form ror Сомротіхс MAXIMAL FLows IN NETWORKS 


Nodes 1 2 j N Labels P 
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flow 


the second stage. We choose the row of lowest index and repeat the 
above process. 

Thus, in a finite number of steps, we either assign a positive value to 
бу. or we find it impossible to label dy. When ôy is assigned a value, the 
flow can be increased. If ôy cannot be assigned a value, the flow is 
maximal, 

Suppose that we have assigned a value to dy. Let Yy = t. We are 
ready to construct a new tableau. First we determine the gag in the path 
from source to sink. These elements are easily discovered in the tableau. 
The last row was reached from row t since Yy = t. Thus the element 
gen in row tis опе of those to be changed. Its new value is бу = gin — ên. 
Suppose that for row t, Y, = $. The next element to be changed is then 
gsi in row s; and its new value is fst = gst — бу. If Y, = r, the following 
element is grs, ete. One element in row 1 will also be changed. This com- 
pletes the sét of elements which are decreased in value by óy. For each 
element gas which is decreased by бу, the element gga is increased by бу 
(an increase in flow in one direction increases the excess capacity in the 
opposite direction). All other elements are unchanged in value. After 
the new tableau is constructed, the whole process is repeated. Since 
бу > 0, we must, in a finite number of steps, reach the maximal flow 
(assuming that it exists), i.e., the condition under which a value cannot 
be assigned to бу. : 

For any network, dj; and dy; are zero for all j since the flow is always 
away from the source and into the sink. Hence gj; = %1j, 9) = 23%, 


346 NETWORK FLOWS [cuap. 10 


and Ygj; = gn; = total flow in the network. Thus, to find the total 
flow in the network, we can add all the numbers in column 1 of Table 
10-1, or we can add all the numbers in the last row of Table 10-1. The 
total flow is entered in the lower left-hand column of the tableau. It is 
convenient to number the rows and columns of the tableau as shown 
so that no counting has to be done in the labeling process, and to add a 
column indicating the stage at which the given row was labeled in order 
to eliminate the need to keep in mind which rows were labeled at the 
previous stage. 

In every case, the z;; for the maximal flow can be found from z;; = 
max (di; — gij 0), i = 1,...,N,j =1,... ‚ М. When dij, dj; are not 
both positive, then ту = gji if dij is positive. In such a case, it is helpful 
to number the nodes so that i < j. Then all gji lie below the principal 
diagonal of the tableau. 


10-7 An example of the matrix solution. Let us find the maximal flow 
in the network of Fig. 10-3, using the tableau form rather than the dia- 
gram of the network. The initial tableau is given in Table 10-2. For 
this tableau, gij = di; since ж) = 0. 

At stage 1, only à», 53 can be assigned positive values. Beginning with 

‚ Tow 2, positive values can be assigned to 54, ôs. For example, ôs = 
min (025 = 1, êg = 6) = 1. No value сап be assigned to any addi- 
tional êj by means of row 3. Thus at the end of the second stage, all 


rows but the last have been labeled. Moving to row 4, we see that . 


TABLE 10-2 — TABLEAU 1 


Labels 
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дав > 0 so that a value can be assigned to ёв, and 
ôs = min (046 = 8, à4 = 4) = 4. 


Thus the flow can be increased by 4 along the traced path. To find the 
goa in this path, we begin by noting that node 6 was reached from node 4; 
hence we put a circle around g4s. By examining Y4, the label on row 4, 
we see that it was reached from node 2, and we place a circle around g24 
in row 2. Then Уз shows that node 2 was reached from node 1, and 012 
is circled. We have reached the source, and the path is completed. In 
the next tableau, each of the circled elements is decreased by êg = 4. 
For each element gas decreased by ôe, the element gga is increased by ё. 
We place a square around these elements in the tableau, as shown in 
Table 10-2. A new tableau is now constructed in which the circled ele- 
ments of the previous tableau are decreased by ôs = 4 and the elements 
enclosed by a square are increased by ôs = 4. All these elements are 
unchanged in value. The new tableau is given in Table 10-3. The value 
of the flow is 4. 

The entire procedure is repeated for the new tableau. The remaining 
tableaux are given in Tables 10-4 through 10-6. The reader might re- 
examine Figs. 10-3 through 10-7 and note that there is a tableau corre- 
sponding to each figure. However, the order is slightly different. The 
flow in each branch is found from z;; = max (di; — gij, 0). The zij are 
the same as those obtained previously. 


TABLE 10-3 — TABLEAU 2 


Nodes 1 2 3 4 5 6 Labels | Stage 
1 = 2 0 0 праи ар 
2 4 - 2 0 H 0 2 1 1 
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TABLE 10-4 — TABLEAU 3 
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TABLE 10-6 — TABLEAU 5 


10-8 Proofs. As yet, we have not actually proved that the above tech- 
nique does indeed yield a maximal flow, We now wish to carry out the 
proof. It is quite clear that by construction a feasible flow is obtained 
at each stage. The capacity restrietions are always satisfied, and there 
is conservation of flow at all nodes except the source and the sink, This 
latter property follows from the fact that when the flow is changed in 
any branch, it is changed in the entire series of branches which form a 
path from the source to the sink. It remains to prove that the final flow 
is maximal, We shall begin by defining a cut in a network. 


Cur: A cut in a network is a collection of oriented branches such that every 
oriented path from source to sink contains at least one branch in the cut, 


For any network, the number of different cuts is finite. If we sum the 
capacities of the branches in a eut, it is clear that a maximal flow cannot 
be greater than this sum, because the flow in every path is limited by the 
branch of lowest capacity, and every oriented path contains a branch of 
the cut. r 

Consider the final tableau that is obtained by means of the computa- 
tional procedure described in Section 10-6, It is not possible to label 
the sink in this tableau. Denote by L the set of nodes which have been 
labeled and by U the set of nodes which cannot be labeled. Then 1 € L* 
and N € U. Let us study the set W of all oriented branches (1, }) which 
join labeled nodes to unlabeled nodes, that is, i € L, j є U. For each 


* The source can be considered to be in the set of labeled nodes by definition. 
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of these branches g;; = 0, for otherwise node j could be labeled. This 
set of branches W is a cut in the network. If an oriented path from 
source to sink did not contain even one branch from W, every node in 
this path would have to be in L or every node would have to lie in U, 
because if the sets of nodes in L and in U were both nonempty, a branch 
of the path would be in W. However, 1 € L and N € U; this results in 
& contradiction. 

Finally, let us show that the sum of the capacities of the branches in 
W is equal to the flow in the network. Note first of all that the manner 
in which the g,; are defined determines that dij фу = — (dji — gji). 
Because of the conservation of flow at any node other than a source or 
sink, Eq. (10-1) must hold. But z; = di — fir, and zy = dy; — gy = 
—(dik — ga) when a net flow is permitted in only one direction, and 
Tij — tjj = dij — gi; when a net flow is allowed in both directions, 
Hence (10-1) can be written for any node i which is not the source or 


sink as 
N 


У) d4—9)0-0  i-22..,N—1 (039) 


j=l, ji 


When i = 1, that is, 7 is the source, then 


N N 
P (à; — д) = D> Tij (10-14) 


j-1, jmi j-2 


80 that the summation is the total flow. 


Consider 
N 
»» У) (0—0), 
EL j=l, jæi 
the first summation being over the labeled nodes. Because the source 
is one of the labeled nodes we see that 


N N 
2) (da — gu) = DD z = total flow. (10-15) 
i€L j=1, j*i j=2 
If j € L and i € L, then d;; — gi; and dji — gj; both appear in the 
summation and cancel each other. Hence all terms in the summation 
cancel for j Є Г, and only terms for j € U appear. But when j € U 
and i € L, then gi; = 0. Consequently, (10-15) reduces to 


У) У) dj = total flow. (10-16) 
i€L jeu 


Recall that the set of all branches (7,3), i € L, j € U form a cut in 
the network. Furthermore, the left-hand side of (10-16) is the sum of 
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the capacities of the branches of this eut. We have already noted that the 
sum could not be smaller than the total flow in the network. Here we have 
a case where the sum of the capacities is equal to the total flow. Hence 
the flow must be maximal, and we have proved the validity of the compu- 
tational procedure. 

We have also proved another interesting result which is often referred 
to as the maz flow-min cut theorem. If for each of the finite number of 
cuts that can be made in the network we find the total capacity of the 
cut, i.e., the sum of the capacities of the branches, then the smallest total 
capacity is equal to the maximal flow in the network. 


10-9 A primal-dual algorithm for the transportation problem. In 
Section 8-12, we discussed a primal-dual algorithm for solving the general 
linear programming problem. The same algorithm can be used to solve 
transportation problems and, as expected, some considerable simplifica- 
tions can be introduced. Perhaps the most important simplification is 
that each restricted primal becomes a problem of finding the maximal 
flow in a network, and hence the efficient procedure for finding maximal 
flows can be used instead of the simplex method. The algorithm to be 
considered, which is an adaption of the primal-dual algorithm for trans- 
portation problems, was developed by Ford and Fulkerson [4, 5]. 

When the primal-dual algorithm is applied to the transportation prob- 
lem, it is unnecessary to annex the additional constraint which makes it 
easy to obtain a solution to the dual. We can find immediately a solution 
to the dual of a transportation problem without using this artifice. Recall 
that the dual of the transportation problem is 


Uitv Se, i$—lbh..om o  j—l.eonm 
m n 
maxZ = Y^ aj; + >> divs, (10-17) 
i=l j=1 


where the и;, vj are unrestricted in sign, If we select 


u; = mincij, $E og. yh 
1 
» = min [ey =u), | J =1,--.,2, (10-18) 


then these variables satisfy the dual constraints and are therefore a solu- 
tion to the dual. 

It will be recalled that when we start to apply the primal-dual algorithm, 
only artificial vectors are in the primal basis. When the artificial variables 
are driven to zero, the resulting basic feasible solution to the primal is 
optimal. For the transportation problem, these artificial variables do not 


352 NETWORK FLOWS [снАр. 1 


need to be introduced explicitly. To see this, we note first that as long 
any artificial variables are positive, not all primal constraints will be 
satisfied exactly by the positive legitimate variables, However, since the 
artificial variables are non-negative, the legitimate variables must satisfy 


р 
2526 < $29 a Mh \ 
j1 

DIR SU gena (10-19). 
i=l 4 


We wish to minimize the sum of the artificial variables, i.e., maximize 
the sum of the negatives of the artificial variables. However, the sum of 
the artificial variables is 


Faded I 2. 2D) xij. 
“j 


і 
pen j=l r 

) 

Thus minimizing the sum of the artificial variables is equivalent to 
maximizing 


{ by Zi. (10-20). 


For any restricted: primal, not. nll Tij are allowed to be positive (in! 
general). Let R be the set of indices ij such that the current solution to. 
the dual satisfies the dual constraint ij аз a strict equality. Then the set’ 
of primal variables Q which are allowed to be positive in the restricted 
primal is Q = (z;jij € R}. Because of this restriction, the equations 
(10-19) cannot always be satisfied as strict equalities when (10-20) is 
maximized. However, since we maintain Tj = 0 when u; + vj € су 
(complementary slackness), we are sure that when a feasible solution to | 
the primal is obtained, it will be optimal. 

The problem of maximizing (10-20) subject to (10-19) with only zij 
in Q allowed to be positive can be Solved as a network-flow problem. 
"The appropriate network is shown in Fig. 10-12. We shall find it con- І 
venient to number the nodes.as shown. Note that the directed branches 
(4,3) either have zero capacity (for those z;; which must remain zero) 
or have no capacity restriction. Furthermore, observe that a net flow | 
is allowed only from 7 to j and not from j toi. When the flow in the 
network is maximized, X; туу is maximized, subject to the constraints 
(10-19). 

The usual transportation tableau rather than the tableau format of 
Table 10-1 can be used for maximizing the flow in Fig. 10-12. To see this, 
imagine that we place a circle inside those cells of the transportation tab- 
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Source 


Destinations 


Ficure 10-12 


leau for which z;; can be positive. Since there is no capacity restriction 
on such @ 7), one does not need to record the excess capacities. However, 
we may be able to move from j to 7, that is, impose a flow in the wrong 
direction. The excess capacity gje of (77) is precisely 3; Hence, by re- 
cording the z;; we simultaneously take care of all necessary information 
concerning capacities. 

The transportation tableau is augmented by two additional columns 
and two additional rows. These are used for labeling the nodes in Fig, 
10-12. Since the same number can appear on two different nodes, we 
must distinguish the labels of the origin nodes from the destination nodes. 
Let ôi, Yi be the labels on the origin nodes, and £j, p; the labels on the 
destination nodes. Define 


= ai — у? 27] D 
dg (10-21) 
у= Does j= b.m 


ll 


1 


Tsj 


Then «i is the excess capacity of the branch leading from the source to 
the origin node ї, and х,у is the excess capacity of the branch leading 
from destination node j to the sink. The column giving the a; in the 
transportation tableau will now contain the zi. Similarly, the row giving 
the b; in the transportation tableau will be replaced by a row listing the Tsj. 

Imagine now that we have some flow in the network of Fig. 10-12. 
We represent this flow in the transportation tableau by listing the posi- 
tive туу values and by giving the xi, х,у. Note that positive z;; must occur 
only in circled cells, i.e., in cells for which х;у is allowed to be positive. 
We begin the labeling process by finding those rows of the tableau for 
which zi > 0 (ie. there is excess capacity in the branch leading from 
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the source to 7). For these rows, set à; = дї, Y; = s (we have come fro; 
the source). For the network under consideration, the sink cannot, 
labeled at the first stage. 

Beginning with the labeled row of lowest index 7, we look for columns j 
such that cell (7, j) will contain a circle (that is, z;; can be positive), 
For each such j, we set £j = тї, p; = i. The value of p; tells us that we 
came from row î during the process of labeling column j. In labeling 
columns, we only require that cell (2, j) be circled; we do not require 
that 2;; in the given flow be positive. If a column has been labeled su 
that z,; > 0, we apply the procedure (discussed below) for modif: 
the flow. Otherwise, we move to the labeled row with the second lowest 
index (if there is such a row) and repeat the above procedure. 

Tf, after examining all labeled rows, we find that no column with Xj; 2 0 
has been labeled, we begin the second stage by locating the labeled column _ 
with the lowest index j. Look for cells in this column for which riz > OF 
If row i has not been labeled, we set 6; = min (£j, tij), Y; = j. The value 
of Y; indicates the column from which we came when labeling row 4. 
Note that in going from 7 to i, we are moving in the wrong direction, 
If a flow is to be imposed in this direction, it must be true that xy > бй 
since no net flow is allowed in the wrong direction. We repeat this process | 
for each labeled column j. 

From the rows of the tableau labeled at the second stage, we choose 
the one with the smallest index and look for unlabeled columns j containi ng 
a circle in the row i being considered. If there are such columns, we set 
ij = ôi p; = i.. This is done for each of the rows labeled at stage 2 
until we label à column with sj > 0 or until the labeled rows are 
hausted. If no column with х,у > 0 is labeled at the third stage, we exam- 
ine the columns labeled at the third stage for unlabeled rows having 
ty > 0, ete. Alternately labeling rows and columns, we continue until, _ 
in a finite number of steps, we reach one or the other of the following E 
cases: 4 

(1) No column with z,; > 0 has been labeled, and it is not possible _ 
to label any additional rows or columns, In this ease, the flow is maximal. — 
Now there are again two alternatives: 


(a) S 2$ — 0, | that is DDE уэ aj; 
ij 


i=l i=l 
GS iSo: 
i=] 


If we arrive at (la), we have a feasible solution to the primal transporta- 
tion problem. Since the solution is feasible, it is also optimal, and hence 
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the problem has been solved. If we arrive at (1b), we have not yet ob- 
tained a feasible solution to the primal. In this case, a new solution to 
the dual is found by a method to be described below. This will release 
at least one new <j; which can become positive. A new flow problem 
(restricted primal) is then solved. 
(2) A column with z,; > 0 is labeled. The flow in the network can be 
increased by 
g = min (tsj, £j). (10-22) 


If p; = k, we increase x; by o, that із, Z4; = ть; + с. Assuming that 
үр = r, we decrease т, by о, that is, £j, = ter — o. Then if p, = t, 
we increase ti by o, that is, £,, = tı +0, etc. We alternately increase 
and decrease the branch flows until we come to a label Y, = s, where 8 
stands for the source label. A new tableau is constructed with the new 
values of the туу, and the labeling process is repeated. This iterative process 
is continued until we reach case (1). 

Now we shall discuss the technique for obtaining a new solution to the 
dual when case (1b) is reached. Let I be the set of rows which were 
labeled in the last tableau (where it was impossible to increase the flow) 
and let J be the set of labeled columns in this tableau. Consider the 
number 

h = min (су — uj = vj) > 0, (10-23) 
fas 


where the u, v; are the values of the dual variables for the current solu- 
tion to the dual. The number h > 0 because we consider only j € J; 
this means that the cell (i,j) does not contain a circle, and hence 
ui vj € б. 

` A new solution to the dual is then given by 


h=uth tel, 


a = ui 1€ I, 

1 (10-24) 
$jv:—h JEJ, 
6; = vj j«J. 


We can see that this is indeed a solution by noting that for i € I, j Є J, 
ûi = ш, 0; = vj; hence d; + ûj < су since u +0; S с. For te I, 
j€J, d; = u +h, 0; = v; — h; hence d; + 0; = wi + v; S су. For 
i€ I, j€J, dj = un 0j = vj — h and G+ 0; = uj-Fvj—h S с. 
Finally, fori € I, j € J, à; = u; + h, 0; = vj and û; + 0j = ur + vj +h. 
But by the manner in which A is chosen ú; + 6; < cij from (10-23). 
Hence (10-24) does provide a solution to the dual. 
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"This new solution to the dual is such that @ + 0; = ci; for at least 
one 17 € R; i.e., because of the manner in which h is selected, at least 
one z;j which was previously required to be zero can now become positive, 
Denote the set of indices ij for which the minimum is taken on in (10-23) 
by st; then d, + 0, = e, Any such st € R since we consider only un- 
labeled columns in labeled rows. If st є R, then cell (s, £) would contain 
a circle, and column ¢ would be labeled. 

It is also true that if ij € R and zij > 0 for the optimal solution to 
the restricted primal, then ú; -+ û; = cij for the new solution to the dual, 
This means that the new set of zij which is allowed to be positive includes | 
all those z;; which are positive in the optimal solution to the restricted 
primal. Hence in the new restricted primal problem, the maximal solution 
to the previous restricted primal problem can be used as an initial solution. 
The proof is immediate. For Tjj > О such that € I, j € J, a; + 6; = 
Uy + vj cj. Also for л; > 0 such that i € IL j EJ, û: 4; = u t 
vj = сіу. There cannot be any л; > 0 with ic I, j € J. Neither can 
there be any z;; > 0 with î € Г, j € J because in this case row i could 1 
be labeled. 

The new value of the dual objective function Z is strictly greater than Z. 
Note that. 


Roa Y au; Th) + D bj; —h)+ Yau; + УВ bjvj 


ier ЈЄЈ ier IGS 
=Z+h(Sa— bj (10-25) - 
ier TT NY N 


However, all i for which x, > 0 must be in J; furthermore, for all j € J, 
5524 = b. 
ier 


1f this were not true, the flow could be increased. Recall that for labeled ji 
either there is no circle in cell (i, j), or when there is, z; = 0016 Д 
(otherwise row 7 could be labeled). Then 


38 Уа = 3 bj. 

jeJ ier jes 
But since дї > 0 for at least one tel, 

У a; OB *U Tij. 

ser ЈЄЈ ieI 
Hence 


25a: — db; > 0, (10-26) 


i€l jeJ 
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and 2 > Z; ie., there is a strict increase in the dual objective function 
whenever a new solution to the dual is found. 

The new solution to the dual gives a new set of dual constraints 
R = (їй + ûj = суу) and a new set of zi, Q = (xj € R) which are 
allowed to be positive in the restricted primal. We then solve the new 
restricted primal problem, using the maximal solution to the previous 
restricted primal as an initial solution. Whenever we find a new solution 
to the dual, the value of the dual objective function is increased. Hence 
we must arrive at case (1a), i.e., at an optimal solution to the given trans- 
portation problem, in a finite number of steps. 


10-10 Summary of primal-dual algorithm. Let us summarize the primal- 
dual algorithm discussed in the last section by presenting a step-by-step 
procedure for solving а transportation problem by means of this algorithm. 
All quantities of interest can be represented conveniently in the tableau 
form shown in Table 10-7. At first glance, the tableau form may seem 
exceptionally complicated, but we shall see shortly that it is not. We 
define $;j, Vi; to be 


фу = су — Uy фу eu — w — vj > 0. (10-27) 


The step-by-step procedure for solving the problem can be described 
as follows: 

(1) Begin by constructing a tableau of the form shown in Table 10-7. 
At the start, only the values of the costs appear in this tableau, To obtain 
the initial solution to the dual, we search each row for the lowest cost. 
Then we set u; = min суу for row ? and enter the u;-values in the first, 
column of the tableau. At the same time, we place a circle inside those 
cells for which u; = су. For the columns j for which one or more circles 
appear we set vj = 0. For the columns j for which no circle appears, 
we compute the ф;; for each 1 and enter them into the tableau. Then we 
find vj = mini фу and list the remaining v,-values. At the same time, 
we place a circle into those cells for which vj = $;j. Now we have an initial 
solution to the dual, and we have circles in those cells for which 2;; can 
be positive in the restricted primal. The фуу are computed for the sole 
purpose of obtaining the initial solution to the dual. They are never 
computed in succeeding tableaux. 

(2) We are now ready to solve the restricted primal by the network- 
flow algorithm. Only the хуу in cells containing circles ean be positive. 
It is very easy to obtain an initial flow in the network, and hence there 
is no need to begin with all z;; = 0. Starting in row 1 of the tableau, 
move to the first circled cell provided there is one. For this cell (1, j), 
set туу = min (a, bj. If bj < ai, move to the next circled cell, say 
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TABLE 10-7 
TABLEAU FORM, ror PRIMAL-DUAL ALGORITHM 


Di Do D D, 


(1, k) in row 1, and set 24, = min (a; — bj, by) ete. This is continued 
until all that is available at origin 1 is used or until the destination require- 


first circled cell (2, r), set Tar = min (b, — даз), etc. In this way, 
we obtain an initial flow and the values of ai. Next compute the zs; for 
this flow and enter them in the appropriate row. If Zz; = 0, move to 
step 6. If Da; < 0, move to step 3. ' 

(3) The maximal flow is found next. For the rows with zi > 0, set 
ôi = тш, =з. In each labeled row look for circled cells, It is not 
necessary that in these cells Tij > 0. For columns j containing a circled 
cell in a labeled row 1, set & = бур, = i. Begin with the labeled row of 
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lowest index 7. If a column for which z,; > 0 has been labeled, go to step 
4. If no column with z,; > 0 has been labeled, look in the labeled columns 
for circled cells with z;; » 0. When there are such cells in a row ? which 
has not been labeled, set 6; = min (zij, £j), Y; = j. In the newly labeled 
set of rows look for circled cells in unlabeled columns. Set £j = ô; 
ру = i. If no column was labeled with z,; > 0, try to label additional 
rows, ete. This process can end in only two ways: 

(a) It is not possible to label a column with z,; > 0. In this case the 
flow is maximal. Compute £ = Y, If Ёё = 0, go to step 6. If £ 0, 
go to step 5. 

(b) A column with z,; > 0 is labeled. In this case, go to step 4. 

(4) Let k be the labeled column with zs > 0. In this case, the flow 
can be increased. A new tableau is constructed. We first fill in the zij 
whose values are changed. Let т = min (x, xax). Then if p, = u, set 
Bun tg. Po: If YQ = t, then 2ш = du — 0$ if p, = v, then 2, = 
zy +ø, ete. The values of the z;; in the path are alternately increased 
and decreased. For all other xij, £;; = xij. Note that the solution to the 
dual does not change during the construction of the new tableau. We 
are still optimizing the restricted primal. Return to step 3. 

(5) When it is no longer possible to label a column with z,; > 0, the 
flow is maximal, and we are ready to determine a new solution to the dual. 
Let J refer to the set of labeled rows and J to the set of labeled columns. 
Then for the i € I, j € J, compute у;; and enter the values in the tableau. 
Next find h = min у;;. А new solution to the dual is given by à; = u; + h 
forie І, i= un iZ TI, 07 —v;—h JEJ, 0 = vj, JEJ. A new 
tableau is constructed, and the new values of the dual variables are 
entered in the appropriate row and column. Circle all cells for which 
û; + 0; = суу. Any cell in the preceding tableau with zı; > 0 will contain 
а circle in the new tableau. At least one cell will be circled in the new 
tableau which was not circled in the preceding one. Enter the z;;-values 
from the preceding tableau into the new tableau. 'The optimal solution 
for the preceding restricted primal can be used as the initial solution for 
the new restricted primal. Return to step 3. 

(6) When in the restricted primal Zz = Xz; = 0, we have obtained 
a feasible solution to the given primal problem. Hence it is an optimal 
solution. The corresponding solution to the dual is also optimal. Fre- 
quently minz can be computed most easily from minz — max Z = 
Law; + Xbjv;. The process must terminate in a finite number of steps 
because each time a new solution to the dual is found, Z > Z. 

4 

10-11 The primal-dual algorithm; an example. The computational 
procedure of the primal-dual algorithm can be illustrated by applying 
it to the problem solved in Tables 9-2 through 9-6. The first tableau 
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is shown in Table 10-8. We start the computation by obtaining the Uy 


for the initial solution to the dual. Beginning with row 1, we look for _ 


the lowest cost. It occurs in column 2 and is €12 = 1; hence we place a 
circle inside cell (1, 2) and set u; = 1. In the second row, we find that 
the lowest cost occurs in columns 2, 3; we place a circle into cells (2, 2), 
(2,3). Since the minimum cost is 2, we set из = 2. Similarly, we find 
Ug = l = u4. Next the v; are found. For each column containing а 
cirele we set vj = 0. Thus va = 03 = U4 = vg = 0. For the remaining 
columns 1, 5, we compute the фу and enter the values as shown. In each 
of these columns, we set vj = min; ф;; and place a circle into the cells 
where фуу assumes the minimum value. Thus we place a circle into cells 
(1, 1), (2, 1) of column 1, and (1,5), (2, 5), (4, 5) of column 5. 

Having determined the 213 which can become positive in the restricted 
primal, we are ready to solve the restricted primal. Beginning with 
row 1, we look for circled cells. Cell (1, 1) is circled, and we set түү = 
min (ау, b1) = min (50, 30) = 30. Twenty additional units are available 
at origin 1; hence we move to the next circled cell, (1, 2), in row 1 and set 
Tia = min (a; — 211, ba) = min (20, 50) = 20. We have allocated all 
units available at. the origin; thus we set x} = 0,2,; = 0. and move to 
the second row. Cell (2, 1) is circled, but the requirement of destination 1 
has been satisfied. The next circled cell is (2, 2); since 30 units are still 
needed at destination 2, we set za, = min (30, a2) = min (30, 40) = 30, 
There are 10 more units available at origin 2; so we move to cell (2,3) 
and set x23 = 10. Now no more units are available at origin 2; we set 
22 = 0, л» = 0 and move on to row 3. In row three, there is only one 
circled cell, (3,6). "Thus we set X36 = 11 = by, x3 = 60 — 11 = 49, 
and хв = 0. Finally, going to the first circled cell in row 4, we set 
t44 = 31 уа, 21 = 0, 244 = 40 — 31 = 9. Only 10 units have been 
shipped to destination 3; hence we set 2,3 = 20 — 10 = 10. Similarly, 
2,5 = 30. 

An initial flow has been found. We now begin the labeling process. 
Only one row has zi > 0; it is row 3. We have ôs = д? = 49, Үз = 8 


There is only one circled cell, (3, 6), in row 3. Thus column 6 is the only | 


one that can be labeled, апа & = 49, pe = 3. Now Ts = 0, and no other 
rows can be labeled from column 6. Thus the flow is maximal. However, 
Since Zz = 0, we do not have a solution to the actual primal problem. 
A new solution to the dual must be found. The set J of labeled rows 
contains only row 3, and the set of labeled columns J contains only 
column 6. Fori € I, jg J › We compute the Y;j, which are listed in, row 
3 of Table 10-8. The smallest Vij is 1; thus В = 1. The new solution to 
the dual is given by da2 da 1— 1-15 9 % — o, 1=0 
L= —1, @& = usd 3, 0; — v j » 6. The new solution to the 
dual is given in Table 10-9. , The solution of Table 10-8 is filled. in 


1 


10-11] THE PRIMAL-DUAL ALGORITHM; AN EXAMPLE 361 


since it is the initial solution for the new restrieted primal. There are 
two new cells, (3, 1), (3, 4), for which ú; + ûj = cij; hence these cells are 
circled. All the cells circled in Table 10-8 have ú; + 0; = cij and are 
again circled. Table 10-9 then presents the initial tableau for the new 
restricted primal. 

At the beginning of the labeling process, we note that only row 3 can be 
labeled. Now, however, columns 1, 4, 6 can be labeled. In addition т; > 0 
for column 4. Thus the flow can be increased by с = min (z,4, £1) = 9. 
A new tableau (Table 10-10) is constructed. The value of the z;; which 
changes is #34 = t34 +o = 04-9 = 9. The label on row 3 is s, so 
this is the only changed variable. 

The labeling in Table 10-10 is more complicated. Columns 1, 4, 6 are 
labeled from row 3. All of these have z,; = 0. Examination of column 1 ° 
shows that жуу > 0, and that row 1 is not labeled. Thus we can label 
row 1. From column 4 we can label row 4. Turning next to the newly 
labeled rows, we see that we ean label column 2 which has х,» = 0, and 
column 5 which has z.s = 30. Неге we have labeled a column with 
z,; > 0. The flow can be increased by с = min (z,5, £5) = 30. A new 
tableau (Table 10-11) is constructed. First the values of the variables 
which change are computed. The three variables which must be changed 
by c are easily found. From ps = 1, we know that 215 is increased by с, 
that is, 215 = 215 Fo = 0 + 30 = 30. Now from Y, = 1 we move 
to column 1, and 2), is decreased by 0, that is, £1) = 2,4 — с = 30 — 
30 = 0. Since pı = 3, we move to row 3 and increase x3, by 0, that is, 
dai = хаа +e = 0 + 30 = 30. The value Уз = s tells us that no 
more variables are to be changed. The +, — signs in Table 10-10 indicate 
how the appropriate variables are changed. All other variables remain 
unchanged. 

The whole procedure is repeated in Table 10-11. The flow can be 
increased; this leads to Table 10-12. Here Yi = 0, and hence we have 
a feasible and optimal solution to the given primal problem. This optimal 
solution is not the same as that obtained in Table 9-6. However, since 
zij — су = 0 for cell (1, 5) of Table 9-6, it is immediately obvious that 
the present solution is simply an alternative optimal solution. Both 
methods require precisely the same number of tableaux, but the primal- 
dual algorithm considerably reduces the work necessary for computing 
each tableau. Note that a new solution to the dual had to be determined 
only once. 

The above example indicates that the primal-dual algorithm developed 
in this section may require fewer computations in solving a transportation 
problem than the stepping-stone algorithm or the modification of this 
algorithm which uses the dual variables to evaluate the zij — cij. Ford 
and Fulkerson feel that the primal-dual algorithm may on the average 
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reduce the computational effort by as much as 50%. They base their 
opinion on experience gained in solving a number of small problems by 
means of both algorithms. No extensive studies using digital computers 
have been made as yet, so that no results are available for large-scale 
problems. Ford and Fulkerson think that the primal-dual algorithm will 
be even more efficient for large-scale problems. If these views are correct, 
the primal-dual algorithm should soon replace the standard stepping-stone 
algorithm for solving the transportation problem. 


10-12 Personnel-assignment problems. Personnel-assignment problems 
are special types of transportation problems. In the simplest form, there 
are m personnel categories with a; individuals available in category t 
and n types of jobs are to be filled. A total of b; individuals can be used 
in job type j. A given individual cannot be assigned to more than one 
job. Individuals in a given personnel category may not qualify for each 
of the n types of jobs. Let J; denote the subset of jobs for which individuals 
in personnel category i qualify. It is desired to determine the assignment 
of individuals which fills the maximum number of jobs. 

It is easy to formulate this problem mathematically. Let z;; be the 
number of individuals in personnel category û assigned to job type j: 
Then the constraints on the problem are 


ү: 
Sty Sa, i=l... 

j=l 

4 

Dt Sbn j=l. (10-28) 


tel 
20120, all i,j, zj 0, IE Si 


The z;; must also be integers. We desire to maximize 


z= 2» ^ Tije (10-29) 


j=1 i=l 


The reader will immediately recognize that this problem has the form 
of the restricted primal in the primal-dual algorithm and hence can be 
solved efficiently by means of the network-flow algorithm devised to solve 
the restricted primal. 

In the general personnel-assignment problem, we again imagine that 
there are m personnel categories with a; individuals in category 7. There 
are n job types, and bj individuals are needed in job type j. No individual 
can be assigned to more than one job, but any individual can be assigned 
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to any job. However, individuals in different personnel categories 
have different degrees of efficiency in the same job. Let r;; measure the 
efficiency of an individual from personnel category i working on job typ 
Assume that Xa; = Xb; so that the number of individuals availab 
the same as the number of jobs to be filled. It is desired to find the a 
signment which will maximize the total efficiency. If т; is the number € 
individuals from personnel category i assigned to job type j, it will 
assumed that the total efficiency is 


a т 1 
2] тушу (10-30 

j=l i=l 

Mathematically, we wish to find non-negative integers туу which та 
mize (10-30) and satisfy the constraints 


n 
dtu = а, t= laem, 


j=l 


SBS ИЕ (10- 


i=l 


This is nothing but the general transportation problem. It сап be solvi 
by any one of the methods discussed previously. 

Interestingly enough, the motivation for using the primal-dual a 
rithm to solve transportation problems stemmed from Kuhn’s work | 
personnel-assignment problems which had only one individual in 
personnel eategory [10]. Ford and Fulkerson modified his algorithm. 
the more general case. І 


10-13 Transhipment problem. In our discussion of the transportai 
problem we have assumed that с was the minimum cost of shipping 9 
unit from origin ? to destination j. If there were several routes fron 
to j, all routes were examined to find the one which yielded the lo 
cost. When there are many routes from each i to each j, the task 0 
determining the minimum-cost routes can. be quite a problem. Here We 
wish to study transportation problems with many possible routes 01 
shipping from 7 to j. а 

We shall begin by studying a very special type of transportation prob 
lem with many routes from each origin to each destination, called 
transhipment problem. In a transhipment problem, we imagine th 
there is only a single direct route from i to j. In addition, however, 
is possible for any given origin i to ship to any other origin ?' over a uniq 
route with a known unit cost of shipment, and similarly, every destin 
tion can ship to any other destination over a unique route with a known 
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unit cost of shipment. The generalization of allowing origins to ship to 
other origins and destinations to ship to other destinations makes it 
possible to ship a unit from i to j over a number of different routes. For 
example, a unit could be shipped from 7 to j by first shipping from 7 to 7’, 
then from 7’ to j’, then from j’ to i”, and finally, from 7” to j. Note that 
we now permit shipment over the direct route from 7 to.j in either direc- 
tion. The unit cost of shipment over the direct route from 7 to j need 
not be the same as that from j to i. 

Since any origin or destination can ship to any other origin or destina- 
tion, we shall find it convenient to number the origins and destinations 
sequentially, beginning with the origins. Thus what we would have called 
destination j in the usual transportation problem will here be denoted by 
т + j. This renumbering is necessary to avoid confusion with respect to 
the subscripts. 

In solving a transhipment problem, we wish to find the minimum-cost 
shipping schedule that will meet the destination requirements. The 
solution to this problem also allows us to determine the minimum-cost 
route for shipping from û to j. As usual, we shall denote by a; the origin 
availabilities and by bm4j the destination requirements (recall that 
depot j is now numbered m + j); we also suppose that Za; = Ура. 
Let ain, ki (k = 1,...,m +n, k z i) be the quantities shipped from 
origin 4 to k and from k to origin 7, respectively; k may be another origin 
or destination. Similarly, let кту т (k = 1,..., m +n, k Am 
- j) be the quantities shipped from Ё to destination j and from desti- 
nation j to k, respectively. 

The net quantity received at destination j (the total quantity received 
minus the quantity shipped to other origins or destinations) must be 
b,4; Similarly, if the destination requirements are to be met, the net 
amount shipped from origin 7 must be a;. In order to meet these require- 
ments, the following constraints must be satisfied: 


= Жулуп A SR Drs C (10-32) 


k=l 
ү kai 
m 
Di Tkym4j — = Lik = bmi ^ J=1,...,n.' (10-33) 
Dem kin 


If we take с; to be the unit cost of shipping from s to £ (s, t= 1,..., 
m + n; s # t), then we wish to find £s > 0 which satisfy (10-32) and 


(10-33) and minimize EU. 
z= D Уу: ТА (10-34) 


=1 t=) 
set 
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Equations (10-32) through (10-34) show that the transhipment problem 
is a linear programming problem. It wiil be noted that if pẹ is the activity - 
vector for Ts then p, = —р„. Hence іп any optimal basic solution тий 
and z, cannot both be positive. This is, of course, intuitively obvious; 
we would never ship in opposite directions along the same route because 
costs could be reduced by making only the net shipment. 

The transhipment problem (10-32) through (10-34) may easily be con- 
verted into the form of a standard transportation problem. First, we add 
to and subtract from each constraint s of (10-32) and (10-33) a variable 
Zs, with cost cs, = 0. Then in (10-32) we set 1 


tig + te = 6;  $—1,...,m, (10-35) _ 
and in (10-33) 
min 
Tom jm4j + a Tatik = 0, j=l... n. (10-36) 
umi 


"The set of constraints (10-32), (10-33) therefore becomes 


min 
S габы wna, i 


= 1,...,m, 
km1 
mtn 
Sala aia: T= 1,-..,2, 
kæ. 
ч (10-37) 
mtn 
ees: t= 1,...,m, 
kel 


mtn : ў 
а m--j б, LS dn. 
ki 


For sufficiently large 6 > 0, the set of constraints (10-37) is equivalent 


to (10-32) and (10-33). Note that the variables z,, take up the slack 
when @ is large. The set of constraints (10-37) can be rewritten 


mtn 
eu s=1,...,m+n, 
4=1 


(10-38) 
mtn 
pate eas t=1,...,m+n, 


s=1 
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9: = 0.4100, s—1l...,m; g,— 0, г = m-Fl.:..,m-m, 
RSG, mS Ее а= +0, = m*- 1, ..., mtn. 


The function to be optimized is 


a — уай. (10-39) 
st 


The set (10-38), (10-39) represents a standard transportation problem 
with m + n origins and m + n destinations. This transportation prob- 
lem is equivalent to the transhipment problem represented by (10-32) 
through (10-34). By solving the transportation problem we find the 
solution to the transhipment problem. 

Note that 6 can be interpreted as the size of a fictitious stockpile at each 
origin and destination which is large enough to take care of all tranship- 
ments. The total quantity shipped is F a;, and hence 0 need not be larger 
than this value. Similarly, the value of z,, is the amount of the stockpile 
not used in transhipments. We shall leave the detailed proof of the 
equivalence of Eqs. (10-38), (10-39) and jus (10-32) through (10-34) 
to the problems, 

Since a transhipment problem is nothing but a transportation problem 
for which we do not compute in advance the minimum cost of shipping 
from i to j, we expect an optimal solution of the transhipment problem 
to have по more than m + n — 1 variables different from zero. When 
we solve the transhipment problem by the equivalent transportation 
problem, we note that this transportation problem can have 2(m + n) — 1 
variables different from zero. However, m + n of these variables represent 
the remaining stockpiles, and hence there are no more than m+n — 1 
variables of interest which are different from zero. 

Given a solution to the transhipment problem, it is easy to find the 
minimum-cost routes to be used in the minimum-cost shipping schedule. 
When 2i,m4; = 0, then the direct route from ? to j is used, and it is 
the lowest-cost route. When there are shipments from one origin to 
another and/or from one destination to another, it is only necessary to 
trace through the shipménts to determine the lowest-cost paths. 


ExAMPLE: Consider a simple transhipment problem involving two 
origins and two destinations. The origin availabilities are 4, 5, and the 
destination requirements are 3, 6. The transportation costs are indicated 
on the network shown in Fig. 10-13. 

Let us first solve the problem as a transportation problem, using only 
the direct routes and ignoring the possibility of transhipment. Table 10-13 
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Figure 10-13 
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gives an optimal tableau for this problem. The minimum cost is 30. 
Next we solve the transhipment problem as an enlarged transportation 
problem. To find the origin and destination requirements, we take 
0 = a, +a, = 9. Then фу = a; + 0 = 13, g = а + 0 = 14, gg = 
ga = 0 = 9; hy = ha = 0, ha = bı + 0 = 12, h, = bz + 6 = 15. An 
optimal solution to the transhipment problem is given in Table 10-14. 
The minimum cost is 21. There is a reduction in cost because it is cheaper 
to ship everything to D», and then from D2 to D;, than ‘to make any 
shipment from either origin directly to Dı: 

In the more general case of a transportation problem with a number 
of routes other than transhipment routes from any given origin to any 
given destination, the simplest approach seems to be to find the lowest 
cost from each origin to each destination, and then to solve the standard 
transportation problem, using the lowest cost of shipment from each т 
to each j. It is interesting to note that the problem of finding the mini- 
mum cost route from a given origin to a given destination can be con- 
sidered to be a transhipment problem. Suppose that an origin 7 is connected 
to a destination j, by a network of routes such as that shown in Fig. 10-14. 
We can imagine that a number is associated with each branch which 
represents the cost of shipping one unit along that branch. (When no 
branch connects two nodes ? and j, we take the cost c;j to be a very large 
number so that no shipment from 7 to j will be considered.) If we number 
the nodes so that the origin is 1 and the destination is N, we have a tran- 
shipment problem such that node 1 is an origin with availability 1, and 
node N is a destination with requirement 1. The intermediate nodes can 
be considered to be either origins or destinations with availabilities or 
requirements of 0. The solution of this transhipment problem yields the 
minimum cost of shipping one unit and the path of minimum cost. The 
basic ideas discussed in this section have been presented by Orden [11]. 
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PROBLEMS 


10-1. Find a maximal flow in the following network, where the numbers 8 
Sociated with the branches are the capacities in the direction shown. 


10-2. Find a maximal flow in the following network. The numbers asso 
with the branches are the capacities in the direction shown. 


® 
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10-3. A network has the following capacitance matrix ||d;;||: 


~ 
К) 


19 8 0 
13 11 
12 


oooocooo! 
oo-ooo! 
Soomi 
ооо оо ч 
о 1 
оо mo m= o 


(a. 


Draw a diagram of the network. Using excess capacity matrices, find a maximal 
flow in the network. 

10-4. Solve Problem 9-4 by the primal-dual algorithm. 

10-5. Solve Problem 9-16 by the primal-dual algorithm. 

10-6. Solve Problem 9-18 by the primal-dual algorithm. 

10-7. Solve the following problem by the primal-dual algorithm: 


Ds Ds Ds De D; Ds Do Dio Dir Diz 


16 


19| 17 
75 | 300 | 100 | 200 | 3100 


10-8. Is degeneracy a problem in the primal-dual algorithm? Why or why 
not? 
10-9. Is it certain that an optimal solution to a transportation problem ob- 
tained by the primal-dual algorithm will always be basic? 

10-10. For the primal-dual algorithm, discuss the relation between the new 
solution [given by Eq. (10-24)] to the dual for transportation problems and the 
new solution [given by Eq. (8-100)] to the dual for the general linear program- 
ming problem. ` 
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10-11. What simplifications are introduced into the two types of personne 
assignment problems discussed in Section 10-12 when there is only one perso 
in cach personnel category? 

10-12. Consider a network with capacity restrictions on the nodes. Ifd 
is the maximum allowable flow through node ї, this means that Xj zj < 
Show how to reduce the problem to one in which there are no capacity resi 
tions on the nodes. Hint: Replace each capacitated node by two uncapacii 
nodes joined by a single branch. 

10-13. Show how to reduce a flow problem involving a number of sources а 
sinks to one in which there is only a single source and sink. | 

10-14. Show that for any 0 > Уа, there is a one-to-one correspondence 
tween the feasible solutions to (10-32) through (10-34) and (10-38), (10-3 
Let x be an optimal solution to (10-32) through (10-34). Show that the 
responding solution to (10-38), (10-39) is also an optimal solution to ( 
(10-39). Thus show the complete equivalence between the two problems, _ 

10-15. Consider a transhipment problem. Let Z(8) be the minimum co 
the equivalent transportation problem when @ is used to compute the 
availability and destination requirements. Prove that if 05 > 01,200) < Z 

10-16, Prove that for any transhipment problem, a value of 6 > 0 can | 
found such that all z, > 0 in any optimal solution to the equivalent 
portation problem. 

10-17, For any transhipment problem, show that if 0* is large enough su 
that all z, > 0 for any optimal solution to the equivalent transportation pro 
lem and Z(0) is the minimum cost for апу 6, then Z(8) = Z(6*) if 0 > 6*. 
means that after the stockpiles have become large enough, the minimum co 
independent of the size of the stockpile. From the results of Problems 10-14 ай 
10-15 deduce that an optimal solution to the equivalent transportation proble 
which has all zr, > 0 is an optimal solution to the transhipment problem. 

10-18, Find as many cuts as possible for the network shown in Problem 10-h 
Find a cut such that the sum of its capacities gives the maximal flow. i 

10-19. Using a labcling procedure rather than the solution of a transhipn 
problem, develop an algorithm for finding the minimum-cost route thro 
network. Use this algorithm and the transhipment formulation to find the 
mum cost of shipment and the minimum-cost path in the network below. W! 
method of solution is simpler? Are there any restrictions on your lab 
algorithm? Y j 
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10-20. Consider a directed network and suppose that a number is associated 
with each branch. The number will be thought of as the time required to move 
through that branch. Develop a labeling procedure for finding the path through 
the network which requires the least time. 

10-21. Show how the task of finding a least-time path in a network (discussed 
in Problem 10-20) can be solved as a transhipment problem. 

10-22. For the network of Problem 10-1, assume that the numbers on the 
branches refer to the cost of shipping a unit along the corresponding branch. 
Find the minimum cost of shipping a unit from the source to the sink. Deter- 
mine the minimum-cost path. 

10-23. For the network of Problem 10-2, assume that the numbers on the 
branches refer to the time required to move through that branch. Find the 
minimum time required to move from the source to the sink. Determine the 
minimum-time path. 4 

10-24. Consider a network which differs from the type discussed in this 
chapter only in that the branches are not oriented, i.e., a net flow in either direc- 
tion along a branch is allowed, and the capacity is the same in both directions. 
Show how to find à maximal flow in such nonoriented networks. Is any change 
needed in the matrix method for finding maximal flows in oriented networks? 
Assume that the branches of the network shown in Fig. 10-3 are not oriented. 
Find a maximal flow for this assumption. How does the solution differ from that, 
obtained in Section 10-3? 

10-25. For the networks discussed in this chapter, show that if the enpacity 
of every branch is a non-negative integer, then the maximal flow will also be a 
non-negative integer’ Thus show that the primal-dual algorithm for the trans- 
portation problem will yield an optimal solution with integral values for the 
positive variables if the aj, bj are positive integers. 

10-26. Find all cuts for the network shown in Fig. 10-3. Which cut corre- 
sponds to the optimal solution of Table 10-6? For this particular example, 
verify the theoretical results of Section 10-8. 

10-27. Find a maximal flow in the network shown in Fig. 10-11. 

10-28. When a new solution to the dual is found in the primal-dual algorithm 
for transportation problems, it will be true under certain cireumstances that 
cells which contained circles in the previous flow problem (restricted primal) 
will no longer contain circles in the new flow problem. When does this happen? 

10-29. Consider a personnel-assignment problem in which six individuals are 
to be assigned to six jobs in such a way as to maximize efficieney. Assume that 
the efficiencies are given by the first six rows and columns of the tableau shown 
in Problem 10-7 where the rows refer to individuals. 

10-30. Develop the form of the primal-dual algorithm for transportation 
problems corresponding to the case where the objective funetion is to be maxi- 
mized. 

10-31. For any feasible flow in a network, show that there exists at least one 
set of oriented paths such that if an appropriate flow is imposed along each 
path, the sum of the flows in these paths gives the total flow in the network. 
Given a matrix of the net flows in each branch, devise an algorithm for finding 
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such a set of oriented paths and the corresponding flows. Note that an о 
path implies that in moving from source to sink we never move along a 
in the wrong direction. Use this result to show why it was necessary E 
sider only oriented paths in Section 10-8. 

10-32. Prove the validity of Eq. (10-26) by finding for the optimal sol 
to the flow problem the capacity of the cut described in Section 10-8. 
Show that the capacity of the cut is 


N Уа+ у) bs 


10-33. Write down the dual of the linear programming problem given 
Eqs. (10-1) through (10-3). 


CHAPTER 11 
SPECIAL TOPICS 


"The time has come” the Walrus said, 
“To talk of many things: 
Of shoes—and ships—and sealing waz— 
Of cabbages—and kings— 
And why the sea is boiling hot— 
And whether pigs have wings.” 


Lewis Carroll. 


11-1 Introduction. In this chapter we shall consider a variety of special 
topics. Some of these deal with the possibility of simplifying the solution 
of linear programming problems in certain cases. Others deal with ways 
of making parameter variation analyses and sensitivity studies. We 
conclude the chapter by briefly discussing the relation between linear 
programming and zero-sum two-person games. 


11-2 Postoptimality problems. Once some linear programming problem 
of practical interest has been solved, two situations may arise which 
require additional computations: (1) With practical problems, it often 
happens that we are not only interested in the solution to the given prob- 
lem, but also desire to know how the solution will change if some of the 
parameters, such as the prices or the elements of the requirements vector, 
are changed. In other words, we wish to perform a sensitivity analysis 
and a parameter study. 

(2) This situation is rather unpleasant, but it occurs nonetheless quite 
frequently. After solving the problem, we may discover that one or more 
of the prices were incorrect, one or more of the b; were wrong, and perhaps 
a decimal point was misplaced in some of the а;;. It may even turn out 
that some variable of interest or some constraint was omitted from the 
problem. Р 

It is the purpose of the following sections to show how to keep to a 
minimum the additional computational effort required to take care of 
the above problems. In many cases, it is not necessary to solve the prob- 
lem over again. A relatively small amount of work applied to the optimal 
solution will suffice. In other cases, however, there is no alternative 
but to go back to the beginning and re-solve the problem. 
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There are six specific problems which we shall diseuss. These can be i 
briefly stated as follows: || 
(1) How much can the price vector c be changed in some specified | 
way before the optimal solution obtained will no longer be optimal? 
(2) For a given change in c, how do we proceed to a new optimal solu- | 
tion if the original solution is no longer optimal? | 
(3) How much can the requirements vector b be changed in some | 
specified way before the optimal solution will no longer be feasible? || 
(4) If a given change in b makes the optimal solution no longer feasible, — 
` how do we proceed to a new optimal solution? 
(5) How can the addition of another variable (vector) be accounted for? | 
(6) How can the insertion of an additional constraint be incorporated. 
into the system? 


at the end of this chapter. It is usually much more difficult to handle 
changes in the matrix elements than to deal with the problems listed 
under (1) through (4). This is especially true for a;; in the optimal 
basix matrix. ` 


I 
| 
Changes in the matrix elements a;; will be discussed in the problems | 


11-3 Changing the price vector. We shall begin by considering prob- | 
lem (1). Assume that we have an optimal basic solution xs = В, 
2 = ¢pXz to the linear programming problem Ax = b, x > 0, max z = CX 
It will be convenient, to imagine that c is changed in the following way: 
We replace c by 

c* — c 4- óf, (11-1) 


where f is some specified, but arbitrary vector, and ¢ is a non-negative 
scalar parameter. We wish to determine the largest value of ф for which 
the given optimal basic solution remains optimal, 

If we denote by zt — cf the values of 2; — c; when с is replaced by 
c*, then the critical value of ¢ is such that any increase in ¢ would make - 
one or more z} — сў negative. Now 


z} — ef = (ca + Фв)у) — cj — of; 
= 2j — 6j + Ф(ву; — fj), 


where fz is the vector which contains the components of f corresponding 
to the components of c in ср. If all fay; — J; are non-negative, then we 
can make ¢ arbitrarily large without destroying optimality. However, 
if one or more fay; — f; are negative and ¢ > 0 is large enough, then 
the corresponding z} — c} will become negative. The critical value 
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z2 
+ 
| ҮП 


Figure 11-1 


ф > 0 is therefore given by 


TE e. min ig а f gy 0, (11-2 
faye — fr j fay; — fj. i Lan iid 
or 


=o И fpyj—f;2 0, allj. 


If ¢ > фе, the current solution will not be optimal when с? is the 
price vector.* The first vector to be inserted into the basis in order to 
maintain optimality as ф passes through ¢, is the a; determined from 
(11-2) (or any one of the a; if a; is not unique) provided that an optimal 
solution exists for ф > ф.. 

The optimal value of z = ctx = cx + фіх varies continuously with ф. 
However, the optimal solution x varies discontinuously as we jump from 
one extreme point to an adjacent extreme point. This is illustrated 
geometrically in Fig. 11-1. 

We start out with the optimal hyperplane denoted by z. Then a vector 
ФЕ is added to the normal с to give a new hyperplane with normal ct. 
As ¢ is increased, the hyperplane becomes tilted, as shown. Extreme 
point 1 is still optimal when ¢ = ф,. At this point, the hyperplane lies 
on one side of the convex polyhedron, and either extreme point 1 or 
extreme point 2 is optimal. When ¢ > фе, extreme point 2 becomes opti- 
mal instead of extreme point 1. Note that the solution x changed dis- 


* In the event that the optimal solution is degenerate it may remain optimal 
for ¢ > фе, even though the optimality criterion is not satisfied. However, if 
we imagine that the problem is perturbed so that degeneracy does not occur, 
then the statement is strictly true. 
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continuously, while z changed continuously. For the example given, ф 
can be increased arbitrarily, and extreme point 2 will remain optimal 
The limiting hyperplane is гу which has f as a normal. 
By means of the above procedure it is possible to determine the optimi 
solutions for the whole range of ф. First фе is determined. The appropria) 
vector is inserted to give a new solution which will be optimal for som 
range of ф, ф. < ¢ € Фф. Then the next vector is inserted, ete.* 
For problem (2) we imagine that c is replaced by ct = c + f. 
new 2; — cj are 
ay eum O — ej fay; — f; 
If one or more z* — cf < 0, it is only necessary to replace the zj 
by the z} — cj and to continue the simplex method. If the re 
simplex method is used, we must change the first row of By’. 
the new first row will be (1, c5 *B^!), we must add (0, #587!) to th 
first row of By! to obtain the appropriate new first row. In addi 
the first component of each a; is changed from —c; to —(cj + fj). | 
In the event that f introduces a radical change into c, we have a com 
pletely different linear programming problem, and it may be better t 
start, from the beginning and solve the modified problem as if it, we 
new one. However, if only a very small percentage of the c; are chang 
it should be easier to continue from the optimal solution for the old: 


11-4 Changing the requirements vector. To solve problem (3), | 
Shall follow the approach used in Section 11-3 and replace b by 


b* = b + or, 


where r is a specified, but arbitrary, vector and 0 is a non-negative В 
We wish to find the largest 0 for which the optimal basis matrix B 
a feasible solution. 

Consider 


xp = B^b'— Bb + oB'r = xg бу, у= Br, (И 


where xg is the given optimal basic feasible solutiont. If all ys > 054 


* We leave a discussion of some of the situations which can arise, such 8 
Ф: = ф. and the appearance of unbounded solutions, to Problem 11-5. 

t Note that here and in several of the following sections it is assumed 
B-! is available. If the revised simplex method was used, В! is immedial 
available. If the simplex method was used, B-! will be available provid 
any artificial vectors needed to initiate the computation were included as 
columns in the tableaux and transformed at each iteration. Otherwise B 
must be computed directly. ^ 
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Figure 11-2 


can be increased without limit, and x} will be feasible and optimal. If 
one or more y; < 0, the critical value б, (when the first zh; goes to zero) is 


б = SS ee min — FB yi < 0. (11-5) 


If 0 > 6, the basis must be changed in order to maintain feasibility. 

What vector should be inserted into the basis as 0 passes through 6,? 
Here we have an ideal application for the dual simplex algorithm. When 
6 becomes slightly greater than б„, we have а basic nonfeasible solution 
with all z; — су > 0. The dual simplex algorithm immediately tells us 
that column v [v determined from (11-5)] should be removed, and (8-69) 
yields the vector (or if the minimum is not unique, any one of the vectors 
yielding the minimum) to be inserted. Proceeding in this way, we can, 
as 0 is increased, develop a series of optimal bases.* 

It may or may not be true that an optimal solution can be found for 
any 0 > 0. This depends on whether r lies in the convex polyhedral cone 
spanned by the activity vectors aj. If r is an element of the polyhedral 
cone, then b* is always in the cone (by the convexity property), and there 
is always an optimal basic feasible solution (it is not possible to obtain 
an unbounded solution by changing b— see Problem 11-17). The final 
basis for which @ can be increased arbitrarily without losing feasibility 
will correspond to an optimal solution where r is used as the requirements 
vector. Such a case is illustrated in Fig. 11-2. 

When r is not an element of the cone spanned by the aj, a point will 
be reached where any further increase of 0 will mean that there is no feasi- 
ble solution. This will be signaled by the fact that all y,; > 0 when column 


* We leave a detailed discussion of this procedure for Problem 11-8. 
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Figure 11-3 


visto be removed. Such a situation is illustrated in Fig. 11-3. ‘The final 
optimal basis will contain as. The largest value of 6 for which there ів 
a feasible solution is 0. At б, zs is the only variable different from zero. 
One vector would presumably leave the basis if 0 became larger than Ё. 
However, there is no vector which could be inserted to yield a feasible 
solution. 


In problem (4), we make an arbitrary change in b, that is, we replace b - 


by b* = b +r. The new basic solution is x} = B~'b*. If it is feasible, 
our task is finished. When one or more z}; < 0, we apply the dual simplex 
algorithm in order to find a new optimal solution, if such a solution exists. 
If b* is radically different from b, it may involve less work to solve the 
whole problem over again, with b* as the requirements vector. 


11-5 Adding variables or constraints. Problem (5), which involves 
the addition of a new variable, is easily handled. If the variable z&41 
with activity vector а, +; and price c,4; is annexed to the system, We 
compute 


Yeu = Bary эрү — ui овушуа — cun (9) 


where B is the basis matrix for an optimal solution to the original problem: 
If 2,4; — Сл > 0, the present solution remains optimal. When 
Zn+1 — Cn41 < 0, we proceed with the simplex method and insert an+1 
into the basis at the next step. 

Finally, problem (6) is concerned with adding an additional constraint 
after an optimal solution has been found. A very important observation 
is to be made at this point: The additional constraint will either cause 4 
decrease in zmax or leave it unchanged. We assume that the new constraint 
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does not introduce any new variables which have nonzero prices. Hence 
we are maximizing the same z that we were maximizing before the new 
constraint was added. Assume that z can be increased by adding the 
new constraint. Note, however, that the new optimal solution satisfies 
the first m constraints as well as the new опе and hence is a solution to 
the original problem. This contradicts the fact that we had an optimal 
solution to the original problem. Therefore, we can write in general 


MAX Zm41 < MAX Zm, (11-7) 


where the subscript indicates the number of constraints. This analysis 
shows that if the optimal solution to the original problem satisfies the new 
constraint, it is also an optimal solution to the enlarged problem. If it 
does not satisfy the additional constraint, then a new optimal solution 
must be found. 

An initial (m + 1) X (m + 1) basis matrix for the system with the 
additional constraint is 


-1 
в, -f e]; s- 8 e], (11-8) 
Y +1 FYB—! +1 


where B is the optimal basis for the original system. The last column is 
the slack, surplus, or artificial vector associated with the new constraint, 
and Y is a row vector containing the coefficients, in the new constraint, of 
the variables in the optimal basis B. From our knowledge of the parti- 
tioned form of the inverse, we see that the y; for the enlarged basis will 
be of the form [у;, aj], and y; = B^ aj. 

If the vector in the enlarged basis is a slack or surplus variable, its 
price is zero and it follows that the z; — c; are unchanged. However, 
the slack or surplus variable is negative since the original optimal solu- 
tion does not satisfy the new constraint. Consequently, the arrangement 
is such that the dual simplex algorithm may be applied to find an optimal 
solution. 

The additional vector will be an artificial vector if the new constraint 
is an equality. If the artificial variable in the basic solution is negative, 
we can assign à price of zero to the artificial variable and use the dual 
simplex method to drive it out of the basis. If it is positive, we can assign 
to it a price of —M and use the standard simplex method (note that, in 
this case, the z; — cj do not remain unchanged). 

When the new constraint radically alters the nature of the system, it 
may be more efficient to solve the whole problem over again. Similarly, 
if one wishes to add more than one or two new constraints which are not 
satisfied by the present optimal solution (these can be established by a 
quick check), it will be better to start at the beginning. 


E 
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11-6 An example of parametric programming. To illustrate very simply 
the ideas of parametric programming, let us consider the first example 
of Section 4-11 (tableaux in Tables 4-2 through 4-4). This problem was 
also solved graphically in Chapter 1. 

First we shall study a change in the price vector. The price vector is 


‚с = (5, 3, 0,0). Let us find how far c, can be increased without dest roying 


the optimality of the solution. We take f = (1, 0, 0, 0), and ф„ will be 
the allowable maximum increase in cı. From (11-2), 


: 2j — Cj 
= — = fry; — < 0. 
$c thin fy oT’ By; — fj 
For the optimal basis, fs — (0, 1) and Ípys — fa = —0.1053, feya = 
Js = 0.2632. Only one of these values is negative. Consequently, 
з — сз 0.2032 


VEIT Says 7 01053 


to 


-50. 


- 


If cı > 7.50, the basis matrix (a5,a;) will no longer be optimal. 

The vector to be inserted when ф passes through ¢. is аз. Then ag is 
removed and we obtain Table 4-3; however, (fey; — f;) must be added 
to the z; — с, of this table. For the new tableau, fg = (0, 1), and 


fay. = fo = 0.4, {вуз — fy = 0.2. 


Both values are positive, and hence the solution will remain optimal for 

arbitrarily large ¢. This is clear from Fig. 1-1. An increase in c; tends 

to increase the slope of the hyperplane 2 — cx. For $ > $a the extreme 

point on the x-axis is optimal. Авф > oo, z = cx approaches a vertical 

line. The extreme point on the z;-axis will then remain optimal for arbi- 
ily large $. 

е shall consider next what happens when the second component of b 
is increased, i.e., when b is replaced by b + без. To determine 6, the 
largest @ for which the optimal solution of Table 4-4 remains feasible, we 
first compute 


= Boles = 85 = [—0.1579, 0.2632]. 
The second column of В”! is found under a4 in Table 4—4. According to 
(11-5), 


= б, = ming, 285, y < 0; 
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since only y; < 0. Thus the second component of b can be increased to 
25.0, and the given basic solution will remain feasible and optimal. 

Using the dual simplex algorithm, we can find what the basis should be 
when 0 passes through 6,. The variable xg; will become negative, and 
hence аз will be removed. Only one yi; not in the basis is negative. It 
is уза, and thus a4 will enter the basis. Now we can see immediately that 
b + бе; will always be in the cone spanned by a4 = ез and ау. Hence 
the new basis (a4, ат) will be optimal for any 0 > ĝe. 


11-7 Upper and lower bounds. Frequently, a number of constraints 
in a linear programming problem will be of the especially simple form 


0 € z; € dj. (11-9) 


These constraints are called upper bounds on the variables. Such con- 
straints may, for example, represent, sales restrictions (no more than a 
giveh amount of some product can be sold). In a transportation problem, 
they may represent upper limits on the amount that, can be shipped over 
any given route, If there are many such constraints, and if they are treated 
in the conventional manner, they can rapidly build up the size of the basis. 
The special simplicity of these constraints allows us to hope that they 
can be handled without being explicitly introduced into the basis, This 
is indeed the case. 1 
The system of constraints which do not contain the upper bounds will 
be denoted by 
Ax = b, A an m X n matrix. (11-10) 


Assume that, in addition, each variable ту has an upper d; > 0 which 
may be infinite. “Let d = [01 . .- ,d,] and x, = [zo . ухва), where 
ту is the slack variable needed to convert (11-9) into an equation. Then 
the set of constraints including the upper bounds can be written: 


А did T И x20 n20 (11-11) 
I, Eg Ly d 


We shall denote by a = [ау ej] the activity vector in (11-11) corre- 
sponding to ay, and by a; = [0, ej] the activity vector corresponding to 
the slack variable Tsj- й 

А basis matrix Вх for the constraints (11-11) contains т -+ n vectors. 
Let us note that any such basis matrix cannot contain more than m 


vectors a* whose variables are not at their upper bounds. To prove this, 


suppose on the contrary that there are k > m such vectors. Then there 


must be k corresponding slack vectors since the variables are not at their 
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upper bounds. Moreover, there must be n — k additional vectors which 
will be af; if z; = 0 or a} if z; = dj. This gives in total n -+ k > n+ m, 
which is a contradiction. Intuitively, the result just proved should be 
obvious. If no upper bounds exist, there cannot be more than m vectors 
in the basis. Hence, when upper bounds are added, we do not expect 
that, in the enlarged basis, there are more than m variables which are 
not at their upper bounds, 

If there arer < m vectors in B, whose variables are not at their upper 
bounds, then there must be at least m — r vectors in В, whose variables 
are at their upper bounds (since there are only n slack vectors). In addi- 
tion, there must be n other vectors in the basis which will be either slack 
vectors or legitimate vectors. If there аге k > m — т vectors in By 
whose variables are at their upper bounds, then there are m -+ n — k —r 
slack variables in the basis. However, there are only n — k variables 
which are not at their upper bounds. Hence m — r of the slack vectors 
ay; will be in the basis at a zero level; for these slack vectors аў is alsa in 
the basis. Consequently, we can always arrange the vectors in By in such 
а way that the first m columns contain vectors аў such that the vectors aj 
form a basis B for (1 1-10). Columns m + 1,. .. » 2m of В, contain slack 
vectors а, з for the vectors in the first m columns of By. If a vector a; in 
one of the first m columns (say i) of В, is in the basis at its upper bound, 
the corresponding slack vector а,; will appear in column m + i of Be at 
a zero level. The remaining n — m columns of Bẹ will contain either slack 
vectors or legitimate vectors in the basis at their upper bounds. Note 
that any basis В, must contain аў or а}, (or both), j = 1,...,m, since 
either z; or z,; must be positive because d; > 0 for all py 

Thus, any basis matrix Bẹ for (11-11) can be written in partitioned 
form as 


Bs = |І, | t (11-12) 

1 

1 

0 | 
where B is an m X m basis matrix for (11-10). We have included in the 
first m columns of Bx all vectors аў whose variables are not at their upper 
bounds. If r < m variables are not at their upper bounds, we also include 


in the first m columns of В, those m — r columns a7 whose variablés are 
at their upper bounds, for which аз; is also in the basis at a zero level. 
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The inverse of В» can easily be computed by the partitioning method 
of Chapter 2; it is 


B| о QUERI 
1 
| 
Snot о BR 
BS | ша ди E : (11-13) 
o.| 9 | L| 0 
Aha 
о1о орот Ay 


The partitioned form above will be most useful for our purposes. 

We shall now show that one can carry out the simplex computations for 
the enlarged system by only using the tableau form for the constraints 
(11-10). 

If cj is the vector containing the costs for the vectors in Bx, then 


ch = (CB, 0, CR), (11-14) 


where cg contains the prices of the a; in B, and cg contains the prices of 
the a; in R. Then 


cB; = (свВ !,0,ск — СВЕ). (11-15) 


For any vector аў not in В», we have 


zt сз = cB رھ‎ — cj = cgB-^la;j— cy = 2; — cj (11-16) 


since the 1 of the e; in aj enters a row such that the corresponding column 
of с Ву! contains a zero. Thus, when B is the basis matrix for the cón- 
straints (11-10), 27 — cj for a} is the same as zj — су for aj. For any 
slack vector аў; not in the basis, 


dt — can al obe ag; IGE ECEB R) eh es 
(11-17) 


since the 1 in e; must appear in à Tow of aj; corresponding to one of the 
last k columns of By because аў must be in B+. Hence, when B is the 
basis matrix for the system (11-10), we can find the 2%; for a slack vector 
аў by simply taking the negative of гу — с; for aj (which is not in B 
since af appears in one of the last Æ columns of B+). Note that if аў is 
not in the basis В», а, is in the basis, and vice versa. 

Next observe that when aj is not in Bs, 


y= B;aj = [ул -Yj ein where y; = Ва; (11-18) 
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because the 1 in e; does not appear in a row of a} corresponding to 0 
' of the last k columns of Bẹ. The vector €j_m is a unit vector which | 
n — m components with the 1 appearing in the (j — m)th compo 
Then if af, is not in Bs, the 1 in €; must appear in a row corres 
to the last k columns of Bs, and 


уш = Ву aj = [—y5 Yj, nme, (11-1 


where e, ,, , is a unit vector having n — m components and a* 
the rth column of В, if we count from the right (not the left). 1 

Let us now use (11-18) and (11-19) to discuss the problem of insert 1 
а vector into Bx. Equation ( 11-18) shows that af can only replace o 
the first 2m columns of By or its own slack vector aj. If it replaces j 


аў replaces one of the vectors in B, then а? enters the basis and repli 
а vector whose variable is not at its upper bound (or a vector a; whic 
is at its upper bound, for which ar, is also in Bx). If aj replaces om 

the slack vectors a7, in columns m + 1,..., 2m of В», we know | 
one of the variables in the first m columns of B, is now at its upper bound 
This in turn implies that to maintain the form assigned to B+, wem 
rearrange the new basis so that аў replaces the vector a (whose varial 
has reached its upper bound) in the first m columns of Bs. This ve 

a, whose variable is now at its upper bound, should be included amoi 
the last k columns of Bx, and аў, leaves By. Intuitively, a? cannot rep 

one of the last k columns of By because, by assumption, the vectors 
the last k columns have attained their upper bounds, and if a} repla 
such a vector, say af, the resulting collection of vectors would not fo 
a basis since aj, would not be in the collection. 


2m, or až. If аў replaces аў, then ту goes from its upper bound to ze 
When аў, replaces one of the first m columns of Bx, then that variable 
driven to zero while at the same time z; is reduced below its upper boi 
This means that, to retain the hypothesized form of Bx, аў must be placed 
in the column earmarked for аў, (since z; is no longer at its upper Боши 
if аў, was originally supposed to go into column 7, it will now enter colun 
m + r, while the slack vector in column m + r is moved over into the set 
of slack vectors which are at their upper bounds. These transformation 
again yield a type of basis which is in agreement with our hypothesis. | 
If a7; replaces the slack vector in column m - т of Bg, z; is reduced from j 
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its upper bound, and the variable corresponding to column r of B« is now 
at its upper bound. Thus to keep the above type of basis unchanged, 
аў must be moved to column r of the basis, while the vector in column r 
moves to the column originally occupied by a7. 

Finally, let us compute the basic feasible solution x5 for the system 
(11-11): 


x5 = B;z'[b, d] = [Bb — ВТВ, dp — B-'b + B-'Rd, а", d], 
(11-20) 


where d = [dg, d*, d], and dg contains the first m components of d, 
that is, the upper bounds for the variables whose vectors are in the first 
m columns of Вх, d contains the upper bounds for the k vectors (which 
are at their upper bounds) in the last k columns of B+, and d* contains 
the upper bounds for those slack variables in the basis which are at their 
upper bounds. Equation (11-20) shows that the variables corresponding 
to the last n — m columns of Bx are indeed at their upper bounds, If 
Xp = BT» —'B-!Rd = B-'b — Xdjy; then 


xh = [Bb — X dy; dp — Bb + X Фуу» 4", d] 
= (xg, dp — xa, d*, dl. (11-21) 


The sum Y:d;y; is taken over the j for which z; is at its upper bound, and 


аў appears in the last k columns of Bs. 
When a* or a*, is to enter the basis, the criterion for the vector to be 


removed from the basis can be written 


* 
Ви — min (01, 02, d); (11-22) 
uv 
where 
2 RE 2. dBi — EBi inm 
0, = min ——» Ji 0,0, min 7 Div ; 
Viv Viv 
Tiv = Yin if af is to enter the basis and fias = —Yiv if ау, is to enter 


the basis. ‘This follows from (11-18) and (11-19), Note that this com- 
putation can be carried out by using only the information in the tableau. 
form appropriate to the constraints (11-10). j 
It should now be fairly clear how we can solve a linear programming 
problem Ax = b,0 < x < d, maxz = сх, without enlarging the basis 
to include the upper bounds. We only need to know which vectors are 
in the first m columns of B», i.e., which a; are in B, and which variables 
are in the last k columns of Br, i-€., which variables have reached their 
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because the 1 in e; does not appear in a row of аў corresponding to one 
' of the last k columns of Bx. The vector €j—5 is a unit vector which has 
^ — m components with the 1 appearing in the (j — m)th component. 
Then if a7; is not in Be, the 1 in е; must appear in a row corresponding 


to the last k columns of By, and 


Yes = By ajj — [5 Yi enm], (11-19) 


where €,_m_, is a unit vector having n — m components and аў is in 
the rth column of B, if we count from the right (not the left). 

Let us now use (11-18) and (11-19) to discuss the problem of inserting 
а vector into Bx. Equation (11-18) shows that aj can only replace one of 
the first 2m columns of Bẹ or its own slack vector ajj. If it replaces its 
own slack veetor, аў enters the basis at its upper bound. Furthermore, 
since it can never go in at a value greater than its upper bound, a vector 
which enters at its upper bound can always be made to replace its slack 
vector rather than one of the vectors in the first 2m columns of By. If 
аў replaces one of the vectors in B, then a} enters the basis and replaces 
a vector whose variable is not at its upper bound (or a vector až which 
is at its upper bound, for which aj, is also in Be). If аў replaces one of 
the slack vectors až, in columns m +1, ..., 2m of В», we know that 
one of the variables in the first m columns of B+ is now at its upper bound. 
This in turn implies that to maintain the form assigned to By, we must 
rearrange the new basis so that a% replaces the vector aj (whose variable 
has reached its upper bound) in the first m columns of Bx. This vector 
an, whose variable is now at its upper bound, should be included among 
the last k columns of B4, and a7, leaves Be. Intuitively, a* cannot replace 
one of the last k columns of By because, by assumption, the vectors in 
the last k columns have attained their upper bounds, and if a} replaced 
such a vector, say a*, the resulting collection of vectors would not form 
a basis since a7, would not be in the collection, — 

Equation (11-19) tells us that a slack vector ay; may replace one of 
the first m columns of Bs, one of the slack vectors in columns m 6р0) 
2m, or аў. If аў replaces aj, then ту goes from its upper bound to zero. 
When aj; replaces one of the first m columns of By, then that variable is 
driven to zero while at the same time 2; is reduced below its upper bound. 
This means that, to retain the hypothesized form of By, a must be placed 
in the column earmarked for a}, (since 7; is no longer at its upper bound); 
if ayy was originally supposed to go into column 7, it will now enter column 
m -+ r, while the slack vector in column m + r is moved over into the set 
of slack vectors which are at their upper bounds. 'These transformations 
again yield a type of basis which is in agreement with our hypothesis. 
If a7; replaces the slack vector in column т + т of Be, x; is reduced from 


و 


» 
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its upper bound, and the variable corresponding to column r of B« is now 
at its upper bound. Thus to keep the above type of basis unchanged, 
aj must be moved to column ^ of the basis, while the vector in column т 
moves to the column originally occupied by aj. 

Finally, let us compute the basic feasible solution x} for the system 
(11-11): 


x5 = B; '[b, d] = [Bb — B-!Rd, dp — B-'b + B-'Rd, а“, d), 
(11-20) 


where d = [dg, d*, û], and dg contains the first m components of d, 
that is, the upper bounds for the variables whose veetors are in the first 
m columns of By, d contains the upper bounds for the Б vectors (which 
are at their upper bounds) in the last & columns of By, and d* contains 
the upper bounds for those slack variables in the basis which are at their 
upper bounds. Equation (11-20) shows that the variables corresponding 
to the last n — m columns of В» are indeed at their upper bounds, If 
хв = B^!b — B^!Rd = Bob — 3;djy;, then 


* 


xh = [Bb — X dy; ds — Bb + X фу» d”, d) 

= [хв, dp — xp, d^, d]. (11-21) 
The sum Yd;y; is taken over the j for which =; is at its upper bound, and 
аў appears in the last k columns of B+. 


When а* or a*, is to enter the basis, the criterion for the vector to be 
removed from the basis can be written 


* 
s = min (64, 05, do), (11-22) 
Уш 
where 
0,— min Bi, ga 0 6 imn BOT Bt yo < 0; 
Viv "5 М ? —Viv 
Tiv = Yin if af is to enter the basis and Jig» = —Yiv if af, is to enter 


the basis. This follows from (11-18) and (11-19). Note that this com- 
putation can be carried out by using only the information in the tableau, 
form appropriate to the constraints (11-10). 

It should now be fairly clear how we can solve a linear programming 
problem Ax = b,0 < x < d, maxz = cx, without enlarging the basis 
to include the upper bounds. We only need to know which vectors are 
in the first m columns of В», i.e., which a; are in B, and which variables 
are in the last k columns of Bx, ie., which variables have reached their 
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upper bounds. All other necessary information is available if we have a 
tableau giving xs = Bb — У0;у;, all y; = B^ a; and all 2; — с; = 
сву; — cj. We begin as if we were going to solve the linear programming _ 
problem Ax = b, x > 0, maxz = cx. Initially, none of the variables is 
at its upper bound, and hence хв = B~'b. The vector to enter the en- 
larged basis at the first iteration is computed from (11-22). If 6; is 
the minimum, then the vector enters column и of B and column u of Bx. 
We continue as if there were no upper-bound constraints until a point 
is reached where 6; is no longer the minimum in (11-22). 

If 0, is not the minimum in (11-22), then at the next iteration, one 
or more variables will reach their upper bounds. If the minimum in (11-22) 
is d,,.then the variable entering the enlarged basis Bx goes in at its upper 
bound and replaces the slack vector a7,. Looking back to (11-17), (11-19), 
and (11-22), one will see that all Jia) for the slack vector až, removed 
from В, will be contained in the column for a; since Jiu» = —Yiw 
Furthermore, since Zw = c, — Z» we simply change the sign of 2, — б 
to obtain Zs. It is better not to change the sign of y, or' z, — c, in the 
tableau because this can later lead to confusion. To indicate that a* is 
at its upper bound and is in the enlarged basis we place some sort of 
mark on the column corresponding to a, in the tableau. During the entire 
process, B has not changed. 

' Although the vectors іп B have not changed, a vector has entered Be 
at its upper bound, and hence xg must be changed. From (11-21), the 
new Xz, which we shall denote by $5, is 


$5 = хь — Фу, (11-23) 


Thus we subtract d,y, from хв if a? enters By at its upper bound and, 
of course, y, is found in the tableau in the a,-column. After the above 
transformations are made, we again compute min Bim 0}, 23 — c; ЖШ 
Note that, since a7 is in Bẹ, a, is the vector not in Bx; 80 we use cy — 2p 
rather than  — cy; thus a different vector from až, will be selected to 
enter the enlarged basis. 

Let us next consider what to do when the minimum in (11-22) is 42, and 
aj should enter the enlarged basis and replace опе of the slack vectors 
in columns m + 1,...,2m of Bẹ. This means that some vector a* in 
one of the first m columns of В, whose slack vector is to be replaced 
reaches its upper bound. We noted previously that the hypothesized form 
of В, will be maintained if a7 replaces až, and a* is moved to one of the 
last k columns of Bẹ. Hence if 0; shows that хви reaches its upper bound, 
then a, should replace column u of B. Tf, on the introduction of ay, more 
than one za, reach their upper bound, select any one of them and remove 
it from B. Place a mark on the column corresponding to aw, the vector 
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removed, to indicate that it is now at its upper bound. Then transform 
the tableau to obtain the yj, XB, 2j; — Су appropriate to the new basis 
matrix B. 

Now we must subtract Ф.У from XB since х, is at its upper bound. 
Note that, to obtain zs» for аў; (which has just left Bx), we change the 
sign of zu — Cw (20 — Cw will be negative, even though aw has just been 
removed from B, because Yuy is negative; thus Ze» will be positive as 
desired). 

It is quite possible that some variable zq which is in Bẹ at its upper. 
bound will recede from its upper bound at a subsequent iteration. This 
will be signaled by ах, entering the basis. In the standard simplex tableau, 
this will be evidenced by the fact that the vector to enter the basis will 
be a column marked to indicate that the variable is at its upper bound. 
When the minimum in (11-22) is dọ, then aj, replaces а. The vectors 
in В do not change. We simply add Фу, to Хв since zq has been driven to 
zero, remove the mark from column g, and go on. 

If the minimum in (11-22) is 01, then to maintain the hypothesized 
form of Bx, a7 rather than az, must replace the appropriate vector in B. 
Hence, we carry out the transformation of the tableau and then add 
dyy, to Xp, since х, is no longer at its upper bound. Because a, is in column 
т of B, y, = er, and we simply add d, to zp; (before d, is added, 2р» is 
negative since уға < 0). 

Finally, if the minimum in (11-22) is 02, some variable £w in B, say in 
column 7, reaches its upper bound when аў, is inserted into Br. Thus 
а} should be placed in column r of B, and the vector in column т should 
be included in the last k columns of Bx, i.e., it should be removed from B. 
We therefore insert a, into B, remove column r of B, and transform to 
a new tableau, We place a mark in column а„ to indicate that cw is at 
its upper bound. Then we add d, to tar since z, is no longer at its upper 
iu and, in addition, subtract duy from Хв since z, is now at its upper 

ound. 

We have explained above how we can handle upper bounds without 
enlarging the basis to include them. The rules we have presented are 
rather complicated at first reading and, no doubt, appear more compli- 
cated to the reader than they really are. It may be desirable to summarize 
the way in which the computations are carried out. 

We wish to solve the problem Ax = 0,0 € x < d, max = cx. To 
do this we use the tableau form for the problem Ax = b,x > 0, maxz = 
€x so that the upper bounds are not, explicitly introduced. Let B be a 
basis matrix for Ax = b. Write, as usual, y; = Bo'a;, 2j — = 
сву; — cj. However, хв is defined differently; we write Xg = Bob — 
Xdjy;, where the sum dy; is taken over those variables which are at 
their upper bounds, but are not in B. Of course, хв gives the values of the 
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variables in B which, together with the variables which are at their upper 
bounds, satisfy Ах = b. The computational procedure is as follows: 

I. Compute z? — c, = min (27 — cj), 2} — ej < 0, where iS 
£j — суй 2j = 0, and 2? — ej = e; — zj if zj = dj. If all z¥ — c; > 0, 
the solution is optimal, and we go to step III. Otherwise the vector a, 
enters B, or reaches its upper bound, or goes from its upper bound to zero 
at the next iteration. Go to step II. 

II. Compute the minimum in Eq. (11-22). There are six cases to be 
distinguished : : 

(1) Assume zr, is not at its upper bound; if z, is at its upper bound, 
this will be indicated by a mark on column a. In this case, there are three 
alternatives: 

(a) The minimum in (11-22) is 01, and column т of B is to be re- 
placed. Insert a, into column r of B and transform the tableau in the 
usual way. Return to step I. 

(b) The minimum in (11-22) is 65, and the minimum is taken on for 
Zar (if the minimum is not unique, choose any r for which the minimum 
value is taken on). Then a, replaces column r of B which contains, 
вау Aw. Transform as usual to obtain a new tableau. Mark column 
aw to indicate that z, is at its upper bound. After transforming the 
tableau, subtract duy, from the хв. Return to step I. 


that z, is at its upper bound, B is unchanged. Subtract d,y, from xp 

and write these numbers in the Xa-column. Return to step I. 

(2) If z, is at its upper bound as indicated by a mark in column v, we 
have the following possibilities: 

(a) The minimum in (11-22) is 6;, and column rof B is to be replaced. 
Transform the tableau as usual and add d, to the new хр, Remove 
the mark from column v, indicating that z, is no longer at its upper 
bound. Return to step I. 

(b) The minimum in (11-22) ів @2, and the minimum is taken on at 
i= r. Replace column r, вау aw, by a, and transform the tableau in 
the usual way. Remove the mark from column a, and place à mark 
on column ay. Add d, to тв, and subtract duYw from the хр so ob- 
tained. Return to step I. 

(c) The minimum in (11-22) is d,. Remove the mark from column v 
and add dy, to x». Go to step I. 

III. When all z? — c; > 0, an optimal solution has been obtained. 
bci ui gives the values of the z; in B. The marked columns have 
zj d; 

Lower bounds are much easier to handle than upper bounds. They can 
be accounted for at the outset, and it is not necessary to worry about them 
thereafter. Assume that, along with other constraints, there are some 
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additional constraints representing lower bounds on the variables. These 
lower bound constraints are of the form 


0<0;< ту. 1011-24) 


Again it is unnecessary to increase the size of the basis to include con- 
straints of this simple form, 
Let us define the variables 


zy = 23 — 93 or x = Ato,  F=l...,m (11-25) 


Then 7; > 0, and туа, becomes Туа, + gjaj, We now consider a new 
system for which the variables are Ту. The matrix A of the coefficients 
for this system is the same as that for the original system, However b 
is replaced by 

n А 

b — 98. (11-26) 

ji 
Thus to convert a system Ax = b, 0 € g < x, into a system without 
lower bounds, we treat the equivalent system 


Ax=b—Ag, X20 х=Х+Е. (11-27) 


11-8 The capacitated transportation problem. It is much easier to 
treat, upper bounds on the variables for a transportation problem than 
it is for the general linear programming problem. The primal-dual algo- 
rithm yields a very efficient way of handling transportation problems with 
bounded variables, i.e., so-called capacitated transportation problems. 
We shall now present the modification of the primal-dual algorithm which 
was developed by Ford and Fulkerson [7] to solve capacitated trans- 
portation problems. 

Consider the capacitated transportation problem 


n 
zu = an fml...,m 
ji 


EE jäl 
zh Po (11-28) 


0 < Tij < dij, all i,j, 
minz = E epris 
a 


As usual, we assume that Уа = Убу: It is no longer true, however, 
that this condition guarantees a feasible, and hence an optimal, solution. 
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Because of the upper bounds, there may not be any feasible solution. 
"There does not seem to be any simple way of determining at the outset 
whether there is a feasible solution. Of course, the primal-dual algorithm 
will ultimately indicate whether or not a feasible solution exists. 

To convert (11-28) into the form of the primal used in the primal-dual 
algorithm, z;; < dj; is replaced by zi; + Ti; = dij where zi; is a slack 
variable. Then the dual problem can be written 


Ui tojt wij Sey, айту, 


ш; < 0, (11-29) 
^ n 
max Z = ami + >> bp; + DY diwi 
izi jai 17 


where the u; v; are unrestricted in sign. 

Consider now any solution to the dual problem (11-29); let R be the 
set of indices ûj for which u; + 0; + wi; = c;; in the given solution to 
the dual, Also, let 5 be the set of indices 4j for which wij < 0. As before, 
this solution to the dual defines a restricted primal problem for which 
each zij, {j Є R, is allowed to become positive, while all 213,1) € R, must 
remain zero. However, we now have the additional restriction that those 
xip j € S, must be zero, i.e., for ij € S, Tij = dij These conditions 
maintain complementary slackness so that г = Z, and consequently, 
when we have found a feasible solution to (11-28), it will be optimal.* 

The restricted primal problem to be optimized (just as for the standard 
transportation problem) can be solved as a network flow-problem. It 
has the form [note that when the primal is written in the form (8-98) 
no artificial variables are needed on the upper bound constraints]: 


D tij S a, i= 1,...,т, 
i 


Уз € b, j21l...,n, 
£ (11-30) 


0<х;<а4„ айту, 
74 —0, GER, zi = 0 GES, 


max J) xij. 
o 


* We assume that the solution to the dual has the property that if ij € S, 
then GER. It ij € S and ij € Р, complementary slackness requires that 
Zij = 0 and шу = 0. This would unnecessarily complicate the solution of the 
restricted primal. Fortunately, there is no difficulty in finding solutions and 
maintaining solutions such that if ij Є S, then ij E€ В. 
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The appropriate flow network js again that 
of Fig. 10-12. However, the branch (2.7) now 
has a capacity restriction d;;. This network- 
‘flow problem can be solved by means of the 
tableau format of Table 10-7, provided that we , 
change slightly the information contained in Figure 11-4 
each cell. Now we must also have the d;; (and 
for the purpose of finding a new solution to the dual, the w;;). Let us then 
replace each cell of Table 10-7 by a cell of the form shown in Fig. 11-4. 

We no longer have room to carry along the equivalents of $;; and yj, 
although they are being used. 

To begin the solution of the restricted primal, we set xi; = dij for 
1) Є S. It is desirable to place a mark in these cells to indicate that during 
the process of optimizing the restricted primal, the 2;;, ij € S, must 
always be at their upper bound. For those rows and columns for which 
ûj Є S, compute the z,;, zi in Table 10-7. Now place a circle in the 
cells for which 77 € R but ij € S, and determine an initial flow (as de- 
seribed in step 2 of Section 10-10), taking account of the upper bounds 
On тү, i.e., no х;; can be greater than its upper bound. 

‘The labeling process used to determine whether the flow can be іп 
creased is almost identical to that used for the standard transportation 
problem. The only difference arises in labeling the columns. Suppose 
that row i has been labeled, and we are trying to find the columns which 
сап be labeled from row i. We look for cells containing circles in row û, 
whose variables are not at their upper bounds. For these columns, we set 


ij = min [dij — 25, Y]. (11-31) 


When attempting to label new rows from labeled columns, we must, 
also remember that we never consider marked cells which signify that 
these variables must remain at their upper bounds, i.e., we never impose 
а flow in the wrong direction through those branches whose variables - 
must, remain at their upper bounds. With these minor modifications, tho 
procedure for maximizing the flow is precisely the same as for the standard 
transportation problem. У 

Once the restricted primal has been optimized, we compute { = lu. 
Tf t = 0, we have an optimal solution to the capacitated transportation 
problem. If on the other hand ¢ > 0, a new solution to the dual with 
Z > Z can be found. 

; Let us finally consider the determination of the néw solution to the 
dual for the case where t > 0. In the development of this algorithm, 
Probably the most difficult task was to determine how a new solution to 
the dual could be obtained. For the tableau giving the optimal solution 
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to the restricted primal, let J denote the set of labeled rows, and J the 
set of labeled columns. Then it is easy to show that 


a= [th iel, 


u; ih by en 

Wij — hu tel, j€J, andiij ER, 

bij = iwi + h, i€ I, jeJ, and ies, (11-32) 
Wij, all other i,j, 


is a solution to the dual provided that 


h = min [min (ci; — ш; — vj — wij), min |w;]] > 0, (11-33) 
е E 
Jen es 


or В = оо if there are no elements in the sets of (11-33); The reader is 
asked to prove this in Problem 11-22. 

It is also true that the new solution to the dual is such that 2 > Z 
and the increase in Z is proportional to h. Thus if h = oo, the dual has 
an unbounded solution, and the primal has no feasible solution. The 
proof that 2 > Z can be made in a way similar to that used in Section 
10-9. It is slightly more difficult, however, and is left for Problem 11-24. 

If h is finite, the new solution to the dual defines new sets R and 8, 
which will be denoted by Rand S. From the manner in which h is chosen, 
it follows that Ё will contain at least one 1j which was not in R, and/or 
‘at least one ij Є S will not be in S (i.e. at least one primal variable will 
be allowed to recede from its upper bound). Thus a new restricted primal 
is obtained. It is not hard to show that the optimal solution to the previous 
restricted primal may be used as an initial solution to the new restricted 
primal. The reader is asked to prove this in Problem 11-23. 

A suitable initial solution to the dual is 


ш = minc;; vj = min (cj — vj); — wi; = 0, alll i,j. (11-34) 
2 і 


11-9 Secondary constraints. Гог a given linear programming problem, 
it may turn out that a priori considerations suggest that the respective 
slack or surplus variables of a certain group of inequality constraints 
will be different from zero in an optimal solution. Dantzig calls such 
constraints inactive. Active constraints are those which hold in equality 
form for an optimal solution. The constraints that are felt to be inactive 
will be called secondary constraints. They are in no way different from 
any other constraints in the problem except that they are expected to be 
inactive. 
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The above discussion leads us to predict that in an optimal basis the 
slack and surplus variables for the secondary constraints will appear at 
a positive level. Now suppose that before beginning the solution of the 
problem, we drop the secondary constraints. We then have a smaller 
number of constraints and hence a smaller basis. However, since we 
expect that the additional variables of a larger optimal basis which in- 
cluded the secondary constraints will be only slack or surplus variables for 
these secondary constraints, it should be true that the smaller basis will 
yield all the variables that are really needed. 

After obtaining an optimal solution to the smaller problem, we use the 
optimal x and test whether the secondary constraints are indeed satisfied. 
If they are, then we know from Section 11-5 that x is an optimal solution 
to the problem which includes the secondary constraints. Consequently, 
we have finished our task. 

On the other hand, if one or more of the slack or surplus variables of 
the secondary constraints are negative, some change must be made. If 
B is the optimal basis matrix for the system without the secondary con- 
straints, then a basis for the enlarged system which includes the secondary 
constraints can be found by inserting the slack and surplus vectors of the 
secondary constraints as additional columns. This will give a new basis 
of the form 


Byz I 
Ri, fs (11-35) 
Re EO 


The slack vectors give the I, and the surplus vectors the —I (of course, 
Iand —I are not necessarily of the same size). The inverse of this basis is 


Bo о 0 
RB ТЛ TO (11-36) 
RjB^! 0 —I 


Since the prices associated with the additional slack and surplus vari- 
ables are zero, it is clear that the z; — c; remain unchanged, and conse- 
quently the optimality criterion is satisfied for the enlarged system. Just 
as in Section 11-5, we are implicitly assuming that the secondary con- 
straints. do not contain any variables which did not appear in the 
Optimized problem. We thus have, for the enlarged problem, a system 
where the optimality criteria are satisfied and some of the variables in 
the basis are negative. The dual simplex method can now be applied to 
Obtain an optimal solution. 
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When these modifications are made, it is not necessary to add all the 
secondary constraints. If we feel that some secondary constraints which 
are not active for the optimal solution to the smaller problem will remain 
inactive, it is unnecessary to add them into the new basis. Clearly one 
must be very careful in choosing secondary constraints. If a fair number 
of secondary constraints turns out to be active, considerably more work 
may be required to find an optimal solution than if all constraints had. 
been included originally. 


11-10 The decomposition principle for linear programs. Often large 
linear programming problems are encountered in practice which have 
the structure 


A, А» Аз- AL|| xi bo 
Bı 0 0...0 || xo bi 
0 B; 0...0 


(11-37) 
0 0 B3:-+-0 


We here wish to show how such a problem can be solved efficiently by 
means of a decomposition principle which requires the solution of & 
series of linear programming problems whose size is smaller than (11-37). 
The material to be discussed was developed by Dantzig and Wolfe [5, 6]. 

In (11-37) we shall suppose that A; is mo X nj, B; is m; X nj, ху, c; are 
vectors having пу components, b; is a vector having m; components for 
j= l...,r and bo is a vector having то components. If each A; — 0, 
then (11-37) reduces to a set of r smaller linear programming problems of 
the form B;x; = bj, x; > 0, max 2; = сј). 

However, when not all A; = 0, the set of constraints Y: Ах; = bo 
couples the system together, and it is not possible to optimize over а 
particular x; independently of the other x;. Nonetheless, it is possible to 
optimize (11-37) without, using a basis of size Zi оту. We shall now 
Study how this can be done. 

The set of points x; > 0 which satisfies Вуху = b; is a closed convex 
set with only a finite number of extreme points. If the set is strictly 
bounded, it is a polyhedron, and any point in the convex set can be repre- 
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sented as a convex combination of the extreme points. We shall assume 
this to be the case. Denote by xj, the extreme points of the convex set 
of feasible solutions to Bjx; = bj. We shall assume that there are Лу 
such extreme points. Then any feasible solution x; > 0 to Вуху = b; 
can be written 


h, 
ху= Ур Pri 20, k= 1,...,Ь;; 


k=l 
h, 
b» Pry = 1. 


k=l 


(11-38) 


Furthermore, any x; of the form (11-38) is automatically a feasible solu- 
tion to B;x; = bj, since any convex combination of the extreme points 
will be an element of the convex set of feasible solutions. 

Now consider the set of рь; > 0 satisfying the constraints 


h; 
ox by pxjA xi; = bo, 


j=l k=l (11-39) 


h, 
SEE Аже: HE 


k=l 


The set of constraints (11-39) is completely equivalent to the con- 
straints (11-37). Every feasible solution to (11-37) determines a set of 
Pry > 0 which satisfy (11-39), and every set of px; > 0 satisfying (11-39) 
determines a set of x; > 0 satisfying (11-37).* Thus the linear program- 
ming problem (11-37) is completely equivalent to the linear programming 
problem whose constraints are expressed by (11-39), and which seeks to 
maximize the objective function 


h 
vay ss piat (11-40) 


j=l k=l 


If we denote an optimal solution to (11-39) and (11-40) by р k= 
1,...,h;; j= 1,...,r, then an optimal solution to (11-37) is 


he 
x; = pma j=l.. f (11-41) 
k=1 
* Observe that a set of px; > 0 satisfying (11-39) uniquely determines а set 
of x; satisfying (11-37). However, a set of x; satisfying (11-37) may not uniquely 
etermine a set of py; > 0 satisfying (11-39); there will, however, be at least one 
such set of ру. 
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Let us write 
du = At; — fu = ext, all j, k. (11-42) 


Then (11-37) is equivalent to the linear programming problem: 


A. 
Уу у Pride; = bo, 


j=1 k=1 


h, 
Уры == Vyas ут, 
k=l 


Pea > 0, all j,k, 


(11-43) 


maxz = >) /урьу. 
Tk 


The linear programming problem (11-43) has a great advantage over 
(11-37) in that it will, in general, have considerably fewer constraints, 
thus requiring a smaller basis matrix than (11-37). The problem in (11-37) 
has 377 9 m; constraints, while (11-43) has mg + т constraints. However, 
(11-43) will usually involve many more variables, since the number of 
extreme points of the convex set of feasible solutions to Вх; = b; will, 
in general, be considerably larger than the number of components in xj. 
"There would be little advantage to the formulation (11-43) if it were 
necessary to compute every d,;, fij that is, to generate every extreme 
point x;;, before the problem was solved. The above remark is especially 
pertinent if we recall that, for a linear programming problem which has 
many more variables than constraints, it is normally true that a large 
percentage of the vectors never enter the basis and hence are never needed. 

Fortunately, it is possible to generate the d,j fj; as they are used. 
Let us next study the application of the simplex method to (11-43) and, 
at the same time, see how to generate the d; frj- If the constraints of 
(11-43) are written in the form 


2 5; i^ 
25 >= Pkiqkj = b, (11-44) 


j=l k=l 


then 44; = (dry, ej], where e; is the jth unit vector having r components, 
and b = [bg, 17), where 1’ is a sum vector having r components. Denote 
by B any basis matrix (of order mo + r) for (11-44), by р» a vector 
containing the variables in the basis, and by {в a vector containing the 
prices in the basis. 
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Suppose that we have a basic feasible solution pg — Bb to (11-44). 
Let e = (еу, сз) = f£gB7', where e, contains the first mo components 
of о, and сз contains the last r components of є. Then, by (11-42), 


ару — ђ = fsB qu — fij = aide; + 02; — fii 
= (914; — cj)xij +095, (11-45) 


where с; is the jth component of o2. 

To determine whether the given basic feasible solution is optimal, 
we must compute min (zz; — fij) over all К, j. If this minimum is non-. 
negative, the given solution is optimal; otherwise more iterations are to 
be made. Now 


min (ri — fij) = min {min ri — Ја), min (ere — fra) - 
a D 


min (а. — fir)]- (11-46) 


Going back to (11-45), we see that, for a given j, min; (zx; — fij) 
occurs at an extreme point of the convex set of feasible solutions to 
Bjx; = Ьу. Consequently, since each extreme point Ху; is а basic feasible 
solution to Bjx; = bj, ming (xj — frj) is 95; plus the optimal value of 
the objective function for the linear programming problem 


B,x; = by, xj20, minZ; = (eA; — су)ху. (11-47) 


Furthermore, an optimal basic solution to (11-47) gives an extreme 
point xz; for which the corresponding zy; — frj has the smallest possible 
value over k. This ху, can then be used to generate the corresponding 
drj, fej by means of (11-42) and ау. 

Thus to determine min (zk; — frj) over all k, j, we solve r linear pro- 
gramming problems of the form (11-47). Writing Z7 for the optimal value 
of Z; for the jth such problem, we compute 77 + 2}, and 


min (ду — frj) = min (27 02) = 7; toa. (11—48) 
a D J 

Let x*, be an optimal extreme point of (11-47) for j = s. Then if dj, = 
Ах? and 9. = [drs, ej], qz, enters the basis at the next iteration, and 
the price associated with qrs is fre = cx, Thus we have generated the 
Vector to enter the basis. We now return to the problem (11-43) and 
transform to obtain B7", ê, êg, the new values of B7", a, pg. The value 
$ is used to determine a new set of objective functions for the linear 
Programming problems (11-47). These r new problems are solved, and 
thus the next vector to enter the basis in (11-43) is found. This process 
18 continued until an optimal solution is obtained. The theory of the 
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simplex method guarantees that an optimal solution will be obtained in 
a finite number of Steps (provided that degeneracy is treated properly). 

The reader will recognize that the revised simplex method provides an 
efficient way to solve (11-43). To obtain an initial basic feasible solution, 
we use a Phase I in which we add r artificial variables р; and maximize 
the form —Y;p,; In solving the set of problems (11-47), one of the 
Special computational algorithms may often be used to advantage. For 
example, the problems (11-47) may be transportation problems. 

The restriction (made at the beginning of our discussion) requiring that 
the convex set of feasible solutions to B;x; = b; must be a polyhedron 
is unnecessary. The generalization to the case where the convex set is 
unbounded is left to Problem 11-28. 

The computational technique discussed here can also be used to solve 
generalized linear programming problems in which we wish to determine 
not only an optimal vector x, but also optimal activity vectors aj. We 
suppose that the activity vector aj may be freely chosen from the set of 
solutions to the inequalities Ва, X b; It will be assumed that the set 
of a; satisfying these inequalities is a convex polyhedron Ку. Thus aj 
can be freely chosen from the convex polyhedron K;. That is, we wish to 
solve the linear programming problem 


Э 
2 та; = b, x 20, max z = cx, (11-49) 
j=l 

where the a; are not Specified, but instead must satisfy Bja; < bj, 

QUE ДЕ А 

"To cast this problem into the above form, let us denote the extreme 


points of the convex polyhedron representing the solutions to Bja; < b; 
by aj, k = 1, ... , hj. Each point a; in the polyhedron can be written 


h A, 
а; = FSI Pki 20, k=1,...,h;; } pe = 1, (11-50) 
k=l k=l 
and any such point is in the polyhedron. Then we obtain the equivalent 
linear programming problem: 
h, 
Ф 
PI » трак; = b, x 2 0, Pki > 0, al kj; 


j=l k=l р 
Уон = 1, j= Ll... 
k 


maxz — cx. (11-51) 
Now note that 


Y ГА 
mL S Ds » суруу, (11—52) 


i=1 j=1 k=1 
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since X; pr; = 1. Thus defining new variables vy; = pxjz; > 0, we ob- 
tain the linear programming problem 


У\оаб = Ь, n;20, all 6) marz = У) слу. (11-93) 
m ik 4 


If all the az; were known, the simplex method could be applied directly. 
Since they are not, the аў, may be generated as needed by the decomposi- 
tion principle described above. Once an optimal solution is obtained, it 
is easy to determine the x; and pij. Let U; denote the set of positive vy; 
fora given j. Then 


2 = DE v; or zj— 0, and for z;7 0, 
кЄ0; (11-54) 


Finally, the optimal activity vectors to be used at a positive level are 
given by* 
а= Уу рагу (11-55) 
FEU; 


11-11 Summary of computational technique for the decomposition 
algorithm; example. It might be desirable to summarize in a step-by-step 
fashion how the decomposition principle is used to solve a linear program- 
ming problem of the form (11-37). We begin by imagining that the given 
problem is converted to the form (11-43), although initially none of the 
vectors qx; or prices fg; are explicitly available. These are generated as 
needed. The revised simplex method is used to solve (11-43) and, in 
general, a Phase I will be needed to obtain a basic feasible solution. In 
Phase I zero prices are assigned to all legitimate variables, and prices of 
—1 to the artificial variables. An initial tableau is constructed. For this 
tableau, B = I (recall that B is of order r+ mo) рв = b, o = ср. 
Now it is necessary to determine the first, vector to enter the basis. Go 
to step (1). 

(1) Compute cA; and the linear form Z; = (mA; — сх), j = 
1,...,7. In Phase I, о is contained} in the last mo + 7 columns of the 
second row of B3" and, in Phase II, in the last mo + r columns of the 
first row of Bz}. 


* For j such that z; = 0, the ры are undetermined, and for these j, any 
a; in K; will do; of course, only the a; to be used at a positive level are of interest. 

Ї Recall that superscripts or subscripts 2 identify vectors appropriate to 
Standard Form II of the revised simplex method. 
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Solve the set of linear programming problems 
Byx; = bj, xj > 9, minZ;— (oA; — €)X, J= 1,... Bi 
and write 
Zj = min Z;. (11-56) 


(2) Compute Z* + g,; j = 1,...,7, and determine 


Zi + on = min (Z] + e. (11-57) 
J 


"There are two possibilities to consider: 
(à) 27 +o. > 0. If this occurs in Phase I, go to step (4), and if 

it occurs in Phase II, go to step (5). 

(b) Z? + cz, < 0. Go to step (3). 

(3) Let x*, be an optimal solution to (11-56) for j — s. Compute 
d, = AS fre E сахи, and q,, = [d,;, e]. Caleulate ym = B; а; 
where 902 = [—f,,, 0, d] and transform the tableau to obtain the 
new basic feasible solution to (11-43). In addition to the revised simplex 
tableau, maintain a table giving the x in the basis. Return to step (1). 

(4) When in Phase I, Z? +02, > 0 and Ура; = 0, Phase I has 
terminated and Phase IT begins. Tf pa; < 0, there is no feasible solution. 

(5) When in Phase IT, 2 92, > 0, the current basic feasible solution 
to (11-43) is optimal. Using the table which gives the хў, in the current 
basis, compute the optimal x; from 


= 22) PRZE) (11-58) 
EV, 


the decomposition principle is applied, the resulting problem will, in gen- 
eral, continue to be of such a magnitude that a large digital computer will 
be required to solve it, 
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ExaMPLE: Let us solve the linear programming problem 


ху + 4ra + 5s F 2x4 < 7, (11-59) 
221 + 3x2 < 6, (11-60) 
5zi +22 < 5, 
23 < 4, 
2i ЖЗ; ~ (11—61) 


—3zg — 414 < —12, 


x20, 
max z = тү + 8z3 + 5x3 + бга, (11-62) 


using the decomposition principle. The constraint (11-59) will be ab- 
breviated to Ax < 7, (11-60) will be abbreviated to Вх X bi, and 
(11-61) will be abbreviated to Bax? < be. 

In solving any problem by means of the decomposition principle, one 
is faced at the outset with the decision as to how far the problem will be 
decomposed. The constraints for the above problem can be written 


(2) Bi < MI n | i b M (11-63). 
B b 0 B; be 


Ai А; 7 


Xi 

(b) B, 0 | BR E (11-64) 

n? 
0/ BAE be 


If (b) is used, then there will be three constraints when the problem is 
converted to the form (11-43); if (a) is used, there will be two constraints. 
For (a) a single subproblem will have to be solved in order to determine 
the vector to enter the bases; the constraints for this problem have the 
form Bx < b. For (b) there are two subproblems whose constraints have 
the form Bix, < bı, Bax; < be. However, Bx < b is equivalent to the 
two sets of constraints Bix, < bi, Взх < bs. Hence the effort required 
for solving the single subproblem for (a) is the same as that necessary for 
solving the two subproblems for (b). However, (a) has the advantage 
that the main problem (11-43) has only two constraints rather than 
three. Hence we shall use the form (a) in solving the problem. We leave 
for Problem 11-32 a discussion of the relative desirability of various 
degrees of decomposition in solving problems by means of the decomposi- 
tion algorithm. 
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The example to be discussed differs from the standard form (11-37) 
in that inequality signs appear. The subproblem can easily be solved 
graphically. The convex sets of feasible solutions to (11-60) and (11-61) 
are shown in Figs. 11-5 and 11-6, respectively. It is thus unnecessary 
to add slack variables to convert (11-60) and (11-61) to equalities. 
However, we shall add a slack variable 25 to convert (11-59) into an 
equation. Note that zs does not appear in (11-60) or (11—61). 

Denote by ху the extreme points of the convex set of feasible solutions 
to Bx < b. The first two components of each xz аге the components of 
an extreme point of the convex set of feasible solutions to Вуху; < bi 
shown in Fig. 11-5, and the last two components of xz are the components 
of an extreme point of the convex set of feasible solutions to Box < be 
shown in Fig. 11-6. Any feasible solution to Bx < b can then be written 


x= S ри, pe 20, alk,  Y'p,—1. (11-65) 
k k 


If we write dy = Ax}, f, = cx}, Eqs. (11-59) through (11-62) are con- 
verted to the equivalent linear programming problem, 


2i dipr + x5 = 7, 
k 

Dee = 1, (11-66) 
k 


maxz = Dd fror 
Г 


The slack variable z5 appears unchanged in (11-66) because it does not 
appear in Bx < b. 

We shall solve (11-66) by the revised simplex method. Only one arti- 
ficial variable, тв, is needed to obtain an initial identity matrix because 
the activity vector corresponding to zs is ej. A Phase I is required to 
drive xg to zero. In Phase I we wish to maximize —zg. For Standard 
Form II, the initial basis matrix js 


155800008 BO 0 
BO O pti uo. ву: |0 1 0—1 
BOL OSG) pep 
ASR и Оа 0з RET 


Then « = (0, —1) = (01,02). For this case, (c1À — c)x = 0 since 
71À = 0, which follows from сү = 0, and c — 0 because in Phase I 
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the legitimate variables have a price of 0. Hence г}, the value of 2x — fr 
for Phase I, is independent of x, and we can choose any extreme point of 
the convex set of feasible solutions to Bx < b to determine a vector for 
entry into the basis. Let us choose extreme point 1 of Fig. 11-5 and 
extreme point 3 of Fig. 11-6 so that the extreme point for the convex set 
of feasible solutions to Bx < b is xj = [0, 0, 0, 3]. Consequently, 


dı = Ау = 6, fi=18, ai? = [—18,0, 6, 1], 
and 
y = Bz'qi? = [—18, —1, 6, 1]. 


The first tableau is shown in Table 11-1. It is seen that the artificial 
vector is removed at the first iteration, and Phase Iends. In Phase II, 
it is unnecessary to retain the constraint which maintains the artificial 
vectors at a zero level, and hence it is omitted in the remaining tableaux. 

The new tableau is shown in Table 11-2. Here « = (0, 18). Recall 
that we are now in Phase II and that the actual prices are being used. 
Again c, = 0 and gA = 0, so that (FA — c)x = —сх. Therefore, 
to generate the next vector to enter the basis, we must solve the linear 


TABLE 11-1 — TABLEAU 1 


peo go в2 o yo 
0 0 0 0 2-18 
1 0 


T | —1 —1 
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TABLE 11-3 — TABLEAU 3 


[cHAp. 11 


(y BSP po» р”. 
1.287 10.28 19.287 —5.72 
0.0357 —0.2145 0.0357 0.284 
— 0.0357 1.2145 0.9643 0.716 
TABLE 11-4 — TABLEAU 4 
pq BU pg? st 
2.005 5.96 20.00 
0.1250 —0.755 0.1250 
—0.1250 1.756 0.875 
programming problem Bx < b, x > 0, minZ — —cx. Of course, mini- 
mizing —cx is equivalent to maximizing cx. This linear programming 


problem decomposes into the following two problems: B, 


Xi € by, xi > 0, 


max zı + 822; Box X bs, xo > 0, max бгз + 624. These problems are 


easily solved graphically, and it is seen that the solution i 
da = 34, fo = 54, qj? = [—54, 34, 1], yt = [—36, 28 
11-2, it is seen that 0 drives x5 to zero. 


вх = [0, 2, 4, 3], 
, 1]. From Table 


The remaining tableaux are 


given in Tables 11-3 and 11-4. At the next solution of the subproblem, 


the optimal extreme point is x} 


(0.287, —2.85, 1.44, —3.43). Also 90 = [—34, 14, I]. 


= (0, 2, 0, 3], since (c;A — с) = 


Note that qj" 


replaces 47. For the final tableau, ¢ = (2.005, 5.96) and (¢,A — c) = 
(1.005, 0.020, 5.025, —1.990). The solution to the subproblem yields x = 


[0, 0, 0, 3] as the optimal extreme point. However, the 
sponding to xi is in the basis, and hence 4—f;=0 
is optimal. 

From the optimal solution we note that рву = 0. 


vector 41) corre- 
and the solution 


1250 and рвә = 


0.875, Since qs is in the first column and qı in the second column of B, 
an optimal x which solves (11-59) through (11-62) is x = paix? + ppox* 


or x = 0.1250[0, 2, 0, 3] + 0.87510, 0, 0, 3] = 


[0, 0.2500, 0, 3]. 


Observe 


that xı = [0, 0.2500] is not an extreme point of the convex set of feasible 
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Figure 11-5 Figure 11-6 


solutions to Bx, X b. Instead it is the point shown in Fig. 11-5. How- 
ever, хә = (0, 3] is an extreme point of the convex set of feasible solutions 
to Boxy < bs. It is extreme point 3 in Fig. 11-6. 

For the example solved, it would not have been difficult to list all the 
extreme points of the convex set of feasible solutions to Bx € b, which 
would have made it possible to compute at the outset all the qx and fr, 
thus eliminating the need to solve the subproblem at each iteration. How- 
ever, even for this simple problem, considerably more work would have 
been required if this procedure had been adopted. 


11-12 Game theory and linear programming; introduction. There is 
a close connection between linear programming problems and zero-sum 
two-person games. We shall investigate the connection in this and the 
following two sections. Let us begin by giving а very brief description 
of game theory. 

Games of strategy deal with situations where there are conflicts of 
interest between two or more “persons.” A game may be a parlor game, 
such as chess or bridge. More generally, games involve conflict situations 
in economie, social, political, or military activities. Games of strategy 
assume that players can influence the final outcome, and hence that the 
outcome is not controlled purely by chance. By a game we shall mean 
а set of rules for playing. These rules describe the moves, who makes 
moyes, when they are made, what information is available to each of the 
players, what terminates a play of the game, ete. After the termination 
of one play of the game, we imagine that there are certain pay-offs to each 
participant (these are usually thought of as pay-offs in money). If the 
sum of the pay-offs to all participants at the end of the play is zero, then 
the game is called a zero-sum game. Two-person games, as the name 
implies, involve a conflict of interest between two “persons” (persons may 


412 SPECIAL TOPICS [снАР. 11 


be people, companies, countries, etc.). A zero-sum two-person game is 
one in which, at the end of a play, one person gains what the other person 
loses. We shall now restrict our attention to zero-sum two-person games. 

The concept of a strategy is very important in game theory. We can 
imagine that one player writes on a sheet of paper what he will do under 
all possible circumstances at each move in a play of the game. In other 
words, he tells what he will do for each possible move the other player 
might make. This is called a strategy. For a game like chess it would 
be an exceedingly difficult task to write out such a set, of specifications, 
but conceptually, it could be done. For each player there will be a number 
of different strategies (quite often a very large number). We shall assume 
that this number is finite. A set of strategies for a given player covers all 
possible alternative ways of carrying out a play of the game, each strategy 
taking into account all possible moves that the other player can make. 
The number of strategies available to each player does not need to be 
the same. 

In two-person games, it is permissible for nature to make some of the 
moves (i.e., chance determines certain moves) so long as the players can 
influence the outcome. If nature does not make any moves and both 
players choose a strategy, the outcome of the game is strictly determined. 
We shall denote by a;; the pay-off to player 1 if he chooses strategy 7 
and player 2 chooses strategy j. When some of the moves are determined 
by chance, the outcome of the game is not strictly determined, and we can 
talk about the expected outcome only. If player 1 chooses strategy i 
and player 2 strategy j, the expected outcome is determined. The ex- 
pected pay-off to player 1 will also be denoted by a;;. Hence, when player 
1 chooses strategy i and player 2 strategy j, ау represents the pay-off 
to player 1 if the game is strictly determined, or the expected pay-off if 
nature makes some of the moves; а; may be positive, negative, or zero. 

If player 1 has m strategies and player 2 has n strategies, the pay-offs 
aij can be arranged into an m X n matrix 


A = [0:3]. (11-67) 


The rows of this matrix refer to the strategies of player 1, and the columns 
to the strategies of player 2. Row i of A gives the pay-offs (or expected 

: pay-offs) to player 1 if he uses Strategy 7. The pay-off depends, of course, 
on which strategy player 2 uses, A is called the pay-off matrix. When 
we have a list of strategies for each player and a pay-off matrix, then the 
game has been reduced to what is called its normal form. 

Since the matrix A represents the pay-offs to player 1 (the pay-offs to 
player 2 are the negatives of these), then in terms of this matrix, player 1 
is trying to win as much as possible, while player 2.is trying to prevent 
him from winning any more than possible, With this interpretation, we 


^ 
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can refer to player 1 as the maximizing player. It will be assumed that 
each player knows his own strategies and the strategies available to his 
opponent. Furthermore, each player is assumed to be intelligent. This 
means that player 2 will do everything possible to cut down the winnings 
of player 1, and player 1 will do everything possible to get as much as 
possible. 

Now if player 1 chooses strategy 7, he is sure of getting 


min aj, 
j 


no matter what player 2 does.* It then seems wise for player 1 to choose 
the strategy which gives the maximum of these minima, i.e., 


max Tm Qij- (11-68) 


Player 2 is attempting to prevent player 1 from getting any more than 
necessary. If player 2 selects strategy j, he is sure that player 1 will not 


get more than 
max агу, 
i 


no matter what player 1 does, It would then seem reasonable that player 2 
attempts to minimize his maximum loss, i.e., select a strategy, using 


min max ау. (11-69) 
we | t 


If it turns out that there is an element (or elements) aj. such that 
a, = max min ау = min max ай, (11-70) 
i i і 


the game is said to have a saddle point: In such а situation, the optimal 
strategy for player 1 is r and for player 2 is k. The reason that these 
strategies can be considered optimal from both players viewpoints is 
that if player 2 chooses k, player 1 cannot get more than ark, and if player 1 
chooses r, he is sure of getting at least ary. 

Difficulties arise when 


ary = max min а < min max aij = аи. (11-71) 
FE Ln 


Player 1 feels that he should be able to do better than ar, and player 2 
feels that he should be able to cut the winnings of player 1 below au. To 


* If nature makes some of the moves, we should say that his expected winnings 
are at least min а;;. We shall not continue to repeat this stipulation. However, 
the reader should keep it in mind. 


414 SPECIAL TOPICS (cap. 11 


resolve this problem, Von Neuniann introduced the notion of a mixed 
Strategy. The strategies considered up to the present are called pure strat- 
egies. One of these pure strategies is actually used by each player in à 
plày of the game. Consider now the case where the player does not select 
the pure strategy he is going to use, but that the strategy is determined 
by а game of chance. Assume that for player 1 some chance device selects 
strategy ¢ with probability и; 2 0,07, uw = 1, and for player 2 another 
game of chance selects strategy j with probability TESTOR олт 
In this way, chance devices determine the strategy each player will use, 
and neither knows what the other's strategy will be. In fact, a player 
does not know what his own strategy will be until it is determined by the 
chance device. The vector u = 0, lu = 1 (or equivalently v), defines a 
mixed strategy, with u; being the probability of choosing pure strategy i. 

When each player decides to use mixed strategies, we can no longer be 
sure what the outcome of the game will be. It is only possible to speak of 
the expected winnings for player 1. If player 1 uses the mixed strategy 
defined by u and player 2 the mixed Strategy defined by v, the expected 
winnings for player 1 are 


Elu, v) = Wav = шалу. (11-72) 
ij 

The reader should note at this point that uncertainty can ereep into à game 
in two different ways. First, nature may make some of the moves. This 
introduces uncertainty, regardless of how the players choose the strategies 
they are going to use. Then the players may choose their strategies by a 
chance device. This introduces uncertainty into the game even if nature 
does not make any moves. 

We have as yet said nothing about the determination of u, v. This is 
really the central problem in solving games, Player 1 wishes to find a u 
Which maximizes his winnings (and here we must mean his expected win- 
hd For any u he chooses, he is sure his expected winnings will be at 
least 


min E(u, v). 
v 


He then maximizes this expression over u so that his expected winnings 
are at least 


МЇ = max min E(u, v). (11-73) 
Чү UE 


Similarly, player 2 can be sure that the expected winnings of player 1 are 
not more than 


М2 = min max E(u, v). (11-74) 
v u 
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Tf there exist vectors u*, v* such that MT = M3, then we have what 
may be called a generalized saddle point, and in such a case player 1 should 
use the mixed strategy defined by u*, and player 2 the mixed strategy 
defined by v*. The expected winnings of player 1 are MT, and this is 
called the value of the game. Interestingly enough, a u* and a v* always 
exist such that 

М? = max min E(u, у) = min max E(u, v) = M3, 
mee MP (11-75) 
u > 0, 1u* = 1; у> 0, 1у* = 1. 


This is the fundamental theorem for two-person zero-sum games. 

The fundamental theorem of two-person zero-sum games tells us that 
a solution in the above sense always exists. The problem of solving any 
two-person zero-sum game consists then in finding the mixed strategies 
described by u, v and the “value” of the game M* = Mf = M3. 

We shall not attempt to defend the procedure of using mixed strategies, 
nor shall we seek to justify it in any more detail, Having given this very 
brief summary of two-person zero-sum games, we now wish to show that 
the fundamental theorem follows directly from the duality theory of linear 
programming. It will also be shown that linear programming can be used 
to solve zero-sum two-person games, and conversely, linear programming 
problems can be solved as zero-sum two-person games. 


11-13 Reduction of a game to a linear programming problem. John 
von Neumann was the first to show that a zero-sum two-person game in 
normal form can be solved as a linear programming problem. Let us see 
how this is done. 

When player 1 uses the mixed strategy defined by u > 0, Iu = 1, and 
player 2 uses pure strategy j, the expected winnings for player 1 are 


m 
ys аш. 
i=l 
Consequently, player 1 can be sure that his expected winnings will be at 
least M; if there exists a mixed strategy u such that 


Dayu > My —Lb.een«h (11-76) 


i=l 


Player 1 can never expect to win more than the largest value of M; for 
which there exists a u > 0, 1u = 1, satisfying (11-76). 
In matrix form (11-76) becomes 


Аа > Mil, uz0 lu—l (11-77) 
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Without loss of generality, we can assume M 1 > 0, because the nature 
of a game (i.e., the set of optimal mixed strategies) is unchanged when the 
same positive constant is added to each a;;. This is to be proved in Prob- 
lem 11-38. Let us then write 


Ui 


WTM $—1,...,m; then ш; > 0 since Mı > 0. 
(11-78) 
Now (11-77) can be written 
A'w >r, Wal: 
ا‎ ав (11-79) 
Mı 


Here we have a linear programming problem. We wish to find a w > 0 
which satisfies the constraint inequalities, maximizes M,, and therefore 
minimizes 7. 

Following the type of reasoning used by player 1, player 2 is trying 
to finda v > 0, 1v = 1, which will give the smallest M. 2 satisfying 


Dy Seis Мает (11-80) 


j=l 
In matrix form, this is 
Av < М v>0, w=1, (11-81) 


If we introduce the variables x; (and without loss of generality assume 
that М» > 0), 


NO 444 а, (11-82) 


1х = 7- =z. (11-83) 


This is also a linear Programming problem for which we wish to find an 
x 2 0 which satisfies the constraint inequalities, minimizes Ms, and 
therefore maximizes z. i 

We can now make the very interesting observation that the linear pro- 
gramming problems for players 1 and 2 are dual linear programming 
problems. It will be noted that there are always feasible solutions u, v 
since each player can use pure strategies. Furthermore, since there are 


* 


© 
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Mı, Mz >:0, there exist feasible vectors x, w for the primal and dual 
problems. This in turn means that the primal and hence the dual have 
optimal solutions. This follows from the fact that since there is a feasible 
solution, the primal (which can be either of the problems) has either an 
optimal or an unbounded solution. If it has an unbounded solution, the 
dual will have no solution, contradicting the fact that there is a feasible 
solution to the dual. Thus both problems have optimal solutions. 
Hence it must be true that 


maxz = min 7, (11-84) 
or 
MŽ = min M; = max M; = МІ. (11-85) 


In this way we have proved the fundamental theorem of game theory. 
It follows directly from the duality theory of linear programming. To 
see more explicitly that we have proved the equivalent of (11-75), note 
that, because of the linearity with respect to v, we have for a given u, 


min E(u, v) = min [E(u, e1), . . -, E(u, е„һ)], (11-86) 


where e; means that player 2 is using pure strategy j. Then 


М} = max min E(u, v) (11-87) 
u v 


can be determined by finding the largest М; such that there exists a и 
satisfying 


Mı < Bue), Jein "20 wl (85 


This is the same as Eq. (11-76). 

Many different proofs of the fundamental theorem of game theory have 
been developed. The first was given by von Neumann in 1928. It was 
quite complicated and involved the use of Brouwer's fixed-point theorem. 
The first elementary proof was made by Ville in 1938. Other proofs have 
been presented by Nash; Gale, Kuhn, Tucker; Dantzig; Weyl; Shapley, 
Snow; and von Neumann and Morgenstern. 

"The task of solving any two-person zero-sum game involves finding the 
optimal strategies for each player and the value of the game. We have 
Shown that such games can be solved by means of linear programming. 
Indeed, this is one of the most efficient methods for solving games. From 
our previous results we know that the optimal strategies need not be 
unique, However, the set of optimal strategies for each player forms a 
convex set. We also note that optimal mixed strategies u, v need not have 
more than min (n, m) of the u; v; different from zero. 
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11-14 Conversion of a linear programming problem to a game problem. 
Here we shall show how to convert any linear programming problem into 
а zero-sum two-person game in normal form. The discussion follows that 
presented by Dantzig [2]. Consider the linear programming problem 


Dx < d, x 20, maxz = cx (11-89) 
and its dual 
Dw > с’, w > 0, тіп Z = d'w, (11-90) 


From Section 8-3 we know that for feasible solutions x, w to the primal 
and dual, 
cx X d'w, or cx — d'w < 0. (11-91) 


Furthermore, when (11-89) has an optimal solution, then 

maxz = min Z. (11-92) 
Consequently, if x > 0, w > Ocan be found which satisfy the constraints 
in (11-89) and (11-90), respectively, and if 

cx — d'w > 0, (11-93) 


[this is the reverse of (11-91)], then we know that x is an optimal solution 
to the primal and w an optimal solution to the dual. 
We shall now introduce a new variable ¢ > 0 and a new set of variables 
u, v defined by 
x= 1 x vtm (11-94) 
In terms of these new variables, the constraints of (11-89) and (11-90) 
become 


Du — (d < 0, 

ру — t > 0. (11-95) 
To these we append 

cu — d'y > 0. (11-96) 


lu+iv+4=1, (11-97) 


E 
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Consider now the game for which the problem of player 1, when cast · 
into the form (11-77), is 


—Du-+ id > MV, 
D — te’ > MV, 
cu — d'v > M, Е 

[v u, 4] > 0, Цу, u, Û = 1. 


(11-98) 


Here M is to be maximized, and max М = M* is the value of the game. 
If max M = 0, then a solution to (11-98) for M = 0 which has t > 0 
yields optimal solutions to the primal and dual linear programming 
problems [by means of (11—-94)]. 

Now (11-98) can be written in matrix form as 


OD д |у 
р’ 0 —c[|[u|2 Mr. (11-99) 

—d' с 0]Lt 
When this is thought of as the problem of player 1, then this player 
wishes to find a mixed strategy defined by y = [v, u, ¢] > 0, 1y = 1, 


which maximizes M. By (11-77) the pay-off matrix for this game is 
merely the transpose of the matrix in (11-99), i.e., 


Q "p. frag 
ci нд ЧА Е (11-100) 
d —c 0 


Note that (11-100) is a skew-symmetric matrix. A game represented by 
a skew-symmetric pay-off matrix is called a symmetric game because the 
roles of players 1 and 2 can be interchanged without changing the pay-off 
matrix. From this it immediately follows that the value of the game 
M* = 0 (the proof is left for Problem 11-35). Consequently, by means of 
(11-94) any optimal strategy y = [v, u, t], t > 0, for player 1 will yield 
optimal solutions to the primal and dual linear programming problems 
(11-89) and (11-90). If t = 0 for every optimal strategy, we leave it for 
the reader to prove in Problem 11-39 that at least one of the primal 
and dual linear programming problems has no solution. 

In this section, we have shown how game theory can be used to solve a 
linear programming problem, However, this is not an efficient way and 
it is never used in practice. Nevertheless, it is interesting to know that 
the possibility exists. 
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PROBLEMS 


11-1. Consider the following tableau which presents an optimal solution to 
some linear programming problem: 


ĉj 2 3 1 0 0 
Vectors 
CB in b а a2 a3 a4 a5 
Basis 
2 al 1 1 0 0.5 4 —0.5 
3 a2 2 0 1 1 21 2 
2; و‎ 8 0 0 3 5 5 


How much can c3 be increased before the present basic solution will no longer'be 
optimal? Determine the sequence of optimal bases as сз is allowed to increase 
indefinitely. 

11-2. For the tableau presented in Problem 11-1, find the largest value of 
¢ for which the given basic solution is optimal if c is replaced by e+ Ф, 5 
(1, 2, 1, 0,0).. Determine the sequence of optimal bases as > is allowed to in- 
crease indefinitely. 4 

11-3. Assume that for the tableau presented in Problem 11-1, a4, as were, 
in that order, in the initial identity matrix basis. How much can b1 be inereased 
before the optimal solution is no longer feasible? Determine the sequence of 
optimal bases as bı is allowed to increase indefinitely. Will there always be a 
feasible basis? 

11-4. Starting with the tableau presented in Problem 11-1 and with the 
assumption made in Problem 11-3, determine how much 52 can be increased 
before the optimal solution is no longer feasible. Determine the sequence of 
optimal solutions as b2 is allowed to increase indefinitely. 

11-5. In Eq. (11-2) show that if the minimum is unique, then $f > фе, that 
is, the new basic feasible solution is optimal over some interval of ф. On the 
other hand, if the minimum is not unique and a, is chosen arbitrarily from 
among the tied values, then the new solution may be optimal only at фе, that 
is, ф/ = d. Is it possible to choose an ак from among the tied values in such a 
way that ¢ > ¢? ‘Consider as a special case that where ¢ in Eq. (11-2) is 
zero. Is it possible that, as ф is increased, the solution will become unbounded? 
How will this be signaled? If one begins with an optimal solution and reaches 
an unbounded solution, is it ever possible that an additional increase in $ will 
again lead to an optimal solution? 
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How much can each price be changed (increased and decreased) if all others 
are held constant and only a single price is changed at a time, before the given 
solution will no longer be optimal? 

11-6. Consider the following tableau which presents an optimal solution to 
some linear programming problem. By how much can со be changed (increased 
or decreased) before the current solution is no longer optimal? 


11-7. For the tableau presented in Problem 11-6, assume that ag, a7, as 

were in the initial identity matrix basis, in the order given. Determine how much 
each b; can be increased and decreased before the optimal solution is no longer 
feasible. Each b; is changed individually, while the others are held constant. 
4 11-8. Show that if the minimum in (11-5) is unique, then there is an interval 
in 0 over which the new solution is optimal. On the other hand, show that if 
the minimum is not unique and an arbitrary selection among the tied values is 
made for the variable to leave the basis, then the new solution may be optimal 
only at &,. Is it possible to make а choice among the tied values in such a way 
that the new solution will be optimal over some interval of 0? Discuss how the 
sequence of optimal solutions may be traced out as 0 varies from 0 to =. 

11-9. Consider the tableau presented in Problem 11-6. Would the optimal 
solution be altered if the activity vector [2, 0, 3] having a price of 5 were annexed 
to the system? Would the optimal solution be altered if, instead of the above 
vector, the activity [1, 1, 2] with a price of 7 were annexed to the original prob- 
lem? If in either case the solution is no longer optimal, find the new optimal 
solution, 

11-10, Consider the tableau presented in Problem 11-6. If the additional 
constraint 

211 + 312.— 13 + 2r4 + 4x € 5, 


were annexed to the system, would there be any change in the optimal solution? 
Suppose instead that the constraint 


311 + z2 + 213 + z4 + 9z5 < 19 
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were annexed to the system. If in either case the constraint is not satisfied, 
determine the new optimal solution (if there is one). 

11-11. Show how to compute the extent to which a;; can be changed before 
an optimal solution to Ax — b, x 2 0, max z — cx is no longer optimal. As- 
sume aj is not in the optimal basis. When an arbitrary, change is made in aij, 
show what change is necessary to obtain a new optimal solution. 

11-12. Answer Problem 11-11, assuming that aj is in the optimal basis. 
Note that when а, is in the basis, a change in а; may affect the nonsingularity 
of B, the feasibility of the optimal solution, and the optimality, i.c. the z; — cj. 
Fach of these cases must be considered separately. Note that it is rather difficult 
to make changes in ау when a; appears іп the optimal basis. If any such changes 
must be made, it is probably better to solve the entire problem all over again. 

11-13. Given the linear programming problem Ax = b, x > 0, max z = cx. 
Let x? be an optimal solution to this problem. Now imagine that the price 
vector c is replaced by new vector c*. Assume that x* is an optimal solution 
to the new problem (which has the same constraints as the given problem). 
Prove that (c* — c)(x* — х0) > 0. If only a single price c; is changed, that is, 
€* = c+ pej, what is the result of this reduction? Give an economic interpre- 
tation of this result. If price cy is changed and all other prices remain constant, 
what (if anything) can be said about the change in ту, j »* k? Hint: cx? > ex; 
thus схо > cx* or c(x? — x*) > 0. Also, c*x* > cx, so c*x* > Cie 

11-14. If onë had a computer program in which it was possible to parametrize 
only the requirements vector for a parameter study, could duality theory be 
used to take care of parametrizing the price vector? Would this be a feasible 
procedure? 

11-15. Generalize the technique of parametrizing the price vector discussed 
in Section 11-3 to the case where $ can be positive and negative. In other words, 
show how to start out with a given optimal basic solution and determine the 
optimal solutions for all values of ф, or show that the solution is ultimately un- 
bounded. Prove that as varies from — to @, there can be only a finite 
number of different optimal solutions. Show that for all ¢ > $* for some 
ф* > 0, the optimal solution will remain unchanged (or the solution will be 
unbounded). Do the same when the requirements vector is parametrized and 
0 is allowed to vary from — to «. Р 

11-16. Suppose that we have a basic feasible solution to a linear programming 
problem and that there exists a vector aj not in the basis with z; — су < 0 and 
Vi < 0, all cj, i.e., there is an unbounded solution. If we now attempt to change 
the price vector in the way discussed in Section 11-3, under what circumstances 
will it be possible to find а ¢ large enough so that an optimal solution exists? 

11-17. Given an optimal basic solution to a linear programming problem. 
Can one obtain an unbounded solution by changing the requirements vector? 
What is the geometrical interpretation of this result? Hint: Is there a solution 
to the dual? 

11-18. Prove that if a given basic feasible solution to some linear program- 
ming problem is optimal, the same basis vectors will yield an optimal solution 
for р requirements vector which lies in the cone spanned by these basis 
vectors. 
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11-19. Solve the following linear programming problem by means of the 
technique developed in Section 11-7: 


2r; + 322 — т3 + 4r4 + 2:5 = 38, 

zı + 4x2 + 223 — 5r4 + 305 > 7, 

9r; — 2х2 + 4xg + r4 + x5 = 24, 
OS <4, OS 25 < 2, zs, za, z5 > 0, 

maxz = 62; ++ 813 + za + 2r4 + a5. 


11-20. Prove that if Eq. (11-34) is used to determine the initial solution to 
the dual, then the primal-dual algorithm for the capacitated transportation 
problem will never lead to a solution to the dual such that ij € S, ij € R, which 
requires that ду = xij = 0 if complementary slackness is to be maintained. 

11-21. Discuss the restricted primal and the determination of a new solution 
to dual if the initial solution to the dual is taken to be u; = vj = wij = 0, all 
i, j, in the primal-dual algorithm for the capacitated transportation problem. 
Assume that each c;; > 0. 

11-22. Show that in the primal-dual algorithm for the capacitated trans- 
portation problem, Eq. (11-32) is a new solution to the dual if Eq. (11-33) 
holds. Do this by considering all possibilities. Consider the sets ij Є R; ij € S; 
ij € R, ij € S, together with the various combinations of labeled rows and 
columns, Be sure to consider also the constraints ws; < 0. Show how the case 
h = o follows from the nature of the new solution. 

11-23. For the primal-dual algorithm as applied to the capacitated transpor- 
tation problem, prove the following; 

(a) Ifi & I and j & J, ij e R, ij € S in an optimal solution to a restricted 

primal problem, then z;; = 0. 

(b te I andj € J, ij Є R, ij € S in an optimal solution to a restricted 
primal problem, then zi; = dij 

(с) If, after a new solution to the dual is obtained, ij Є. $, then xij = di 
in the optimal solution to the restricted primal which yielded the new solution 

to the dual, d 

(d) If, after а new solution to the dual is obtained, ij 6 R, then z;; = 0 
in the optimal solution to the restricted primal which yielded the new solu- 
tion to the dual. Thus conclude that an optimal solution to the previous 
restricted primal will serve as an initial solution for the new restricted primal. 

11-24. For the primal-dual algorithm as applied to the capacitated. trans- 
portation problem, prove that the new solution to the dual given by Eqs. (11-32) 
and (11-33) increases the value of the dual objective function by 


1| 3, а — 250; + Yo da Уау > 0. 
ier jer igi ier 
Ie 3e 
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Hint: Using the min cut+max flow theorem, show that the maximal flow in the 
network is 


V= Уа – d 256 det Dye. du Some de 


igr ig jes jes 1€1,1jER TES vel 
Єв бєз EI ES 
Thus 
Du- Dh Dat yaw E wo 
ier jel jes Ter je ier, ER 
HES HES 12,0098 


Then use (proof?) 


Y dy Уа Do du t Dew — У, do. 

es хат jel ier ig 
ijEs HES ДЕБ бєз 

11-25. Solve the following capacitated transportation problem. The upper 

numbers in each cell are the costs and the lower numbers are the upper bounds. 


Ds ai 
.30 |.15 | 40) 59 

50 

Ln 75 

10 
EKA HA а 20, 35 


25 Еш] 100 


m 11-6. Assume that in 


11-26. Consider the tableau presented in Proble t 
the following constraints 


Solving the problem which led to the given tableau, 
were originally considered to be secondary: 


42у + 212 — 28 + z4 — 315 < 10, 
огу + Aza 42243 + Gra + 125 < 15. 


Does the optimal solution satisfy these constraints? If not, determine an op- 
timal solution which does satisfy the secondary constraints. 
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11-27. Suppose that one has an optimal solution to some linear programming 
problem and it is desired to remove one of the constraints. Discuss the procedure 
for doing this. 

11-28. Suppose that in using the decomposition principle to solve (11-37), 
we find that the problem В;х; = bj, x; > 0, min Z; = (o)A; — cj)x; has an 
unbounded solution (this of course implies that the convex set of feasible solu- 
tions to Bjx; = b;is unbounded). Denote the columns of B; by gu, а basis matrix 
by H, and H~'g, by tu. Assume that g, was being considered for entry into the 
basis H when the unbounded solution was discovered, ie. t < 0, i = 1,..., 
mj, and that the insertion of g, decreases the objective function. Then note that 
Olge — itha’) = 0 for any 0 where hg; are the columns in the basis. In- 
sertion of g, into the solution will yield for sufficiently large 0 an arbitrarily 
small value of Zi. Now let ap;(j) be the columns of A; corresponding to the 
columns of B; in the basis H and, similarly, let св:(ј) be the prices of c; cor- 
responding to the columns of B; in H. Consider 


d; = 0[a (7) — У !вав(ў); 


f; = olej) — 2 tive ;()] 


for any 0 > 0, and 
qj = [4,0]. 


Show that if q; enters the basis of the connecting problem [Eq. (11-43)] at a 
positive level, z will be inereased. Also show that the increase in z is independent 
of 0; hence 0 can be taken equal.to 1, without loss of generality. Note that a 
zero appears in qy in place of e; since in this case there is no restriction on the 
levelat which q; may enter the basis (why?). Prove that with the above modifica- 
tion to take care of unbounded solutions, an optimal solution to (11-37) will be 
reached in a finite number of Steps in all cases. -How is the optimal x; com- 
puted when there are vectors, such as q; above, in the optimal basis for (11-43)? 
Prove that the final x; is also independent of the choice of 0. Note-that there 
will also be at least one extreme of the convex set of feasible solutions to Bjx; = 
b; in an optimal basic solution to (11-43), since the constraint Lpy = 1 is 
present. Thus show that it is certain that the constraints Bjx; = b; will be 
satisfied. 

11-29. Solve the example of Section 11-11, using the decomposition (b) of 
Eq. (11-64), i.e., the decomposition for which there are three constraints in 
Eq. (11-43) rather than two. 

11-30. Solve the following linear Programming problem, using the decom- 
position principle, 

51 + 322 4 45 + 214 + 625 < 20, 
zı + 722 < 8, . 323 + 924 + 225 > 4, 
471 + 222 < 5, 2213-24425 < 12, all zj > 0, 


maxz = 3zı + 4r2 + 5r3 + 474+ 725, 
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11-31. Discuss the use of the decomposition principle to treat linear program- 
ming problems with upper bounds on the variables. Hint: To what degree 
should the problem be decomposed? 

11-32. Diseuss the relative advantages and disadvantages of using various 
degrees of decomposition when employing the decomposition algorithm to solve 
8 linear programming problem. 

11-33. Discuss in detail how the vector af; which is to enter the basis is 
generated in generalized linear programming problems, where the activity vector 
a; may be freely chosen from the convex polyhedron K;. 

11-34. Find a set of mixed strategies for each player which solves a game 
described by the following pay-off matrix: 


3 5 0 9 6 
A= |2 6 81 2 
RA TAK Dee 


Determine the value of the game. 

11-35. Prove that the value of a game with a skew-symmetric pay-off matrix 
is zero. 

11-36. Prove that for any matrix A 


ar, = max min а; < min max а = ашк. 
Li j i i 


11-37. Show that if ax, ay, are saddle points of the matrix A, then aj», аш 
are also saddle points. 

11-38. Prove that the set of optimal strategies for a game is not altered by 
adding a constant M tò each a;; in the pay-off matrix. How does the value of the 
game change? 

11-39. For player 1, show that if every optimal strategy y = [v, u, 1] applied 
to the game whose pay-off matrix is (11-100) has ¢ = 0, then at least one of the 
primal and dual linear programming problems (11-89), (11-90) has no solution. 

11-40 Prove rigorously that for the situation discussed in Section 11-3, zisa 
continuous function of ф. к 

11-41. Develop the theory of parameter variations and sensitivity analysis 
for transportation problems. What special restrictions must be placed on changes 
in the requirements vector in order to maintain the existence of feasible solutions? 

11-42. For the optimal transportation tableau given in Table 9-6, by how 
much can c12 be changed (increased and decreased) if the basic solution shown 
isto remain optimal. Answer the same question for сз4 and c42. 

11-43. In Eq. (11-12), prove that if Bẹ is nonsingular, then B must also be 
nonsingular. Hint: Note that the first m columns of By will be linearly inde- 
pendent regardless of what B is. However, the first 2m columns of B, will be 
linearly dependent unless B is nonsingular. 

11-44. With respect to the decomposition principle, how is it possible ever to 
have x; = O in an optimal solution if it must be true that Lpr; = 1. 
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11-45. If a problem involving upper bounds is solved by the method discussed 
in Section 11-7, how is the optimal value of z obtained from the information 
given in the tableau? Note that the optimal value of z is not to be found in the 
cell which normally contains the value of the objective function. 

11-46. Consider the convex set of feasible solutions X to Bx < b, where D, 
b are as defined in Eq. (11-63). Show that for any extreme point x* of X the 
first two components of x* are an extreme point of the convex set of feasible 
solutions to Вх, < b, and the last two components are an extreme point of the 
convex set of feasible solutions Box» < bs. Generalize this result to the case 
where B is of order n and can be partitioned so that the only non-null submatrices 
are square submatrices B; of order m; lying on the main diagonal, 

11-47. For the optimal tableau given in Table 4-10, determine how much 
each price can be changed (increased or decreased) if the given basic solution is 
to remain optimal. Change only one price at a time. Cases where two or more 
prices are changed simultaneously need not be considered, Similarly, determine 
how much each b; can be changed if the optimal solution is to remain feasible. 
What effects do these changes have on the optimal value of the objective func- 
tion? 

11-48. Start with the optimal tableau given in Table 4-4 and determine the 
optimal basic solutions (if they exist) for all values of bı, that is, as b, varies 
from —æ to =. Plot the optimal values: of the variables zı, z as a function 
of bi. 

11-49. For the situation discussed in Section 11-3, prove that as $ passes 
through ¢, only a single vector in the basis needs to be changed in order to 
maintain optimality. In certain cases, of course, the new solution may be 
optimal only at ¢.. What is the geometric interpretation of this situation? 

11-50. Using a perturbation technique, prove rigorously that if for the situa- 
tion discussed in Problem 11-5, ф = фе, а basis never need be repeated, Hint: 
Replace c; by c; + €. 

11-51. What simplifications are introduced into the technique discussed in 
Section 11-7 if not all the variables have finite upper bounds? 

11-52. Consider those postoptimality problems discussed in the text which 
require application of the dual simplex algorithm for making the necessary 
modifications. Suppose that the revised simplex method was used to solve the 
original problem. Imagine that za, < 0 is to leave the basic solution. How does 
one obtain the уш; needed in the dual simplex algorithm to determine the vector 
to enter the basis? Recall that When the revised simplex method is used, the 
y; are not computed. Hint: It is possible to compute y,; without computing 
all the components of уу. 


CHAPTER 12 


APPLICATIONS OF LINEAR PROGRAMMING TO 
INDUSTRIAL PROBLEMS 


“Tf you would understand me go to the heights or watershore, 
The nearest gnat is an explanation, and a drop or motion of waves a key, 
The maul, the oar, the hand-saw, second my words." 


Walt Whitman. 


12-1 Introduction. Linear programming is a practical subject. In fact, 
the rapid development of the subject is a result of the many practical 
applications that have been found. In this chapter we wish to discuss 
applications of linear programming to certain problems arising in in- 
dustry. We have chosen an approach which will show, in a general way, 
how linear programming can be applied to a number of specific areas of 
interest. 1 

There exists an amazing variety of problems which lend themselves to 
solution by linear programming. It is important to keep in mind that 
for most of these problems, the linear programming model is only an 
approximate representation of the real world. However, the representa- 
tion is often good enough to yield useful results. In any field of science 
or engineering, it is rare, indeed, to find a model which represents the 
real world exactly. The important question is whether the representation 
is accurate enough to provide valuable information. Naturally, one must 
not become overzealous and attempt to apply some model to situations 
where it is completely inapplicable, just because the model is available. 
This is true for linear programming as well as for anything else. 

We shall begin the discussion by illustrating some ways in which the 
transportation algorithm сап be applied. In industrial applications, the 
transportation method has been used to a much greater extent than 
the simplex algorithm, We shall see that a number of important problems 
which have nothing to do with transportation can be cast into the form 
of transportation problems. 


12-2 The standard transportation problem. We haye already mentioned 
on several occasions that the standard transportation problem seeks to 
minimize those costs incurred from shipping a uniform product avail- 
able in given amounts at m origins to n destinations, where the amount 
at each destination is also specified. For example, a wholesaler in some 
hard-goods item has warehouses throughout the United States. Orders for 
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delivery in the coming month are received from retailers in all fifty states. 
It is desired to set up a shipping schedule for the coming month which 
will minimize the total shipping cost. The quantity of the item available 
in each warehouse is known. 

The example just presented seems simple and straightforward. In 
practice, any number of difficulties may be encountered. Note, for ex- 
ample, that the cost of shipping from each warehouse to every retailer 
must be known. Frequently, one finds that a company does not have 
all this information. Consequently, it must be generated in its entirety 
before the problem can be solved. If several methods of transportation 
and a number of different routes are available for shipping from any 
warehouse to any retailer, it can be quite a task to find the (minimum) 
cost of shipping from each warehouse to each retailer. Furthermore, if 
the problem is to be solved at the beginning of each month, the shipping 
costs must be continually revised to take account of changes in shipping 
rates. 

In using the transportation method, we must assume that the unit cost 
of transportation from a given warehouse to a given destination does not 
depend on the number of units shipped. This assumption would not be 
valid if there were any sizable fixed charges that are independent of the 
quantity shipped. Fortunately, however, the assumption that costs are 
proportional to the quantity shipped is frequently close enough to the 
actual situation so that no serious errors are introduced. Since railroads 
and truckers mix their loads, it is not necessary to fill an entire car or 
trailer. 

It is important to note that for a problem involving, say, 15 origins 
and 500 destinations, the cost of collecting the data for the transport..tion 
costs can run into a considerable amount of money. After these data are 
available, the cost of solving the problem on a large-scale digital com- 
puter could be $1000 or more, since the solution may require up to two 
hours of computer time. This does not include the eost of punching and 
checking the cards which form the input data. Whether this expenditure 
is justified depends on the total transportation costs. Many corporations 
spend as much as 10 million dollars per year on transportation costs alone, 
and a saving of 2 or 3% would amount to $200,000 or $300,000 per year. 
Such potential savings attained with a very small percentage improvement 
easily justify the expenditure of a considerable amount of money for the 
application of linear programming to cases of this kind. 

The task of obtaining the correct answer to a large transportation 
problem оп a digital computer is not so easy as One might think, If there 
are 15 origins and 500 destinations, 7500 costs must be punched correctly 
into cards, A single incorrect punch will lead to an answer which is com- 
pletely wrong. Extreme care must be used to be sure that the cards are 
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correctly punched. Sometimes a mistake is immediately evident when 
the final optimal solution is examined. In other instances, however, it is 
not obvious. Even when the mistake is found, something in the order of 
$1000 of computer time has been wasted since the problem must be rerun 
after the mistake is corrected. If & number of runs are required before 
all mistakes are eliminated, the cost of solving the problem increases 
` sharply. 

It is hardly necessary to dwell on straightforward applications of the 
transportation method. However, before going on, we shall present one 
final example. A transportation problem which has received considerable 
attention from certain segments of industry and from military planners 
is the scheduling of marine tankers [5, 7]. Many tanker-allocation prob- 
lems are very complicated and cannot be reduced to linear programming 
problems. However, if appropriate assumptions are made, especially 
simple problems do become transportation problems. 

One of the simplest examples of scheduling a fleet of marine tankers 
can be formulated as follows: The U.S. Navy has known numbers of 
empty tankers at various ports throughout the world. It will be assumed 
that all tankers are identical. In a given time, known numbers of tankers 
must be available at certain ports throughout the world for loading. 
How should the empty tankers be routed to ports where they are needed, 
to ensure that transportation costs are minimized? 

The cost per day of operating an empty tanker on the high seas and 
its speed are assumed to be given. From a knowledge of the distances 
between ports, it is possible to compute the cost of moving a tanker 
from any given port to any other port.- If any route leads through the 
Suez or Panama canals, the appropriate fee can be added to the cost. 
The problem of routing the tankers clearly requires nothing but a straight- 
forward application of the transportation method. Flood [7] has discussed 
a problem of this type. 


12-3 Production allocation and transportation. Many corporations 
today find that the production costs for a given product vary considerably 
among their different plants. Newer plants have lower production costs 
than older plants. An important problem facing management is that of 
Scheduling production at the various plants in such а way as to minimize 
the total variable costs consisting of the variable production costs and 
transportation costs. 

Suppose that a corporation has m plants throughout the United States 
or the world and n jobbing points to which the product is to be shipped. 
Let a; be the yearly productive capacity of plant i and b; be the amount 
demanded at jobbing point j for the coming year. Let cij be the cost 
of shipping one unit of the product from plant 7 to jobbing point j, and 
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k; the variable cost of producing one unit at plant 7. Furthermore, let zij 
` be the number of units produced in plant 7 for shipment to jobbing point j. 
If precisely the amount demanded is to be produced, we wish to de- 
termine z;; which minimize 
z — У) (са + kitij (12-1) 
ij 


subject to the constraints 


Dry €; voc) AAS v 

i 

DEUS ea Pee (12-2) 
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maus 0, al ij. 
If we write 
dij = ci + ki, (12-3) 


the problem is nothing but the standard transportation problem with 
costs dij. 

In solving this problem, we determine the lowest-cost shipping schedule 
as well as the amount to be produced at each of the plants. The produc- 
tion X; at plant 7 for the coming year will be 


Xi 9 Ута, (12-4) 
j 


where the zi; in (12-4) correspond to an optimal solution to (12-1) and 
(12-2). The optimal production-shipping schedule depends on both the 
manufacturing and shipping costs. 

It is not always easy to determine the variable production costs with 
any great accuracy. There is usually some question as to which costs 
should be considered variable and which should be considered fixed. The 
cost of production is rarely directly proportional to the quantity produced 
over the entire range from zero production to maximum capacity. How- 
ever, this proportionality can be assumed to provide frequently a reason- 
ably good approximation over some limited range of production. Each 
ki is determined for the range within which the particular plant is expected 
to operate. If the optimal solution shows that one or more of the plants 
should be operating at a level quite different from that for which the ki 
were determined, it may be necessary to recompute the К; for these new 
Jevels and re-solve the problem. 

Note that if the variable costs of production are the same at each 
plant, we do not need to include the production costs when solving the 
problem. The optimal allocation will depend on transportation costs 
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alone. Similarly, if there is no excess capacity in any of the plants, i.e., 
the total demand is precisely the sum of the plant capacities, it again 
is unnecessary to include the production costs even though they may 
vary from plant to plant. 

The demands b; for the coming year at the various jobbing points 
will, in general, be based on forecasts. Depending on the nature of the 
industry, it may or may not be possible to make accurate forecasts of 
the demand for the coming year. The worst case is that in which the 
total sales cannot be predicted accurately and it is also impossible to 
estimate accurately the percentage of sales to be expected at each jobbing 
point. If accurate forecasts cannot be made, the problem may have to 
be re-solved later in the year to take account of more recent data. In 
such circumstances, the problem may have to be solved on a quarterly 
or monthly basis rather than once a year. The decision depends on the 
nature of the industry. Some industries (e.g. food-processing industries) 
turn out their entire production during two or three months of the year. 
If their forecasts are bad, they have no opportunity to correct them. 
Other industries which produce the given item continuously through the 
year can alter schedules to a. certain extent, depending on the nature of 
the industry. 

It should be observed that, in general, an optimal solution for a lower 
total level of sales cannot be found by simply multiplying the optimal ay; 
for a higher predicted sales level by some constant factor, because the 
plant which produces at capacity at the higher sales level should, perhaps, 
also do so at the lower sales level and ship to some of the jobbers that 
previously received deliveries from a different plant. In general, the 
decision on which plant should ship to which jobbers cannot be made 
independently of the level of total sales. 


12-4 A case study, To illustrate the application discussed in the pre- 
ceding section by a practical example, we shall briefly discuss a case 
study on which the author worked in cooperation with company personnel. 
No actual data or tableaux will be presented. 

A large food processor with four plants in different, sections of the 
United States produces a number of different food products. The study 
dealt only with a group of twenty packaged foods. Each plant makes all 
twenty products, and the product mix is about the same at all plants. 
Production costs vary considerably among the plants because some are 
more modern than others. There is also excess capacity in the four plants, 
ie., the total productive capacity is greater than the present demand. 

About 2500 jobbing points in the United States receive deliveries from 
these four plants. A plant in Canada and plants overseas take care of the 
foreign business; these were not considered in the study. The company 
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also has some government business. All military orders are processed at 
one plant because it is the only one with special facilities for military 
packaging. The government business was not included in this study, 
and its total volume was subtracted from the total capacity of the plant | 
where government orders are processed. 

'The United States is divided into seventy sales districts, which have 
been in existence for many years. Although it is not a strict requirement, 
the company has followed the policy of having all jobbers in a given 
sales district receive shipments from the same plant. Over the years, - 
the traffic department has been at work, attempting to minimize trans- 
portation costs. In recent years, the disparity between processing costs 
at the various plants increased, and management became worried about 
two problems. First, they were not sure that the sales districts were as- 
signed to the proper plants so that total variable costs were indeed mini- — 
mized. Secondly, they wondered whether the present districts were any _ 
longer suitable. They wished to know how much could be saved if the 
restriction were relaxed that all jobbers in a sales district had to receive 
shipments from the same plant. In other words, they wondered what - 
changes would take place if the cost minimization were based on shipping 
to jobbers rather than to sales districts. This information could be used 
to form new sales districts if the practice of shipping to all jobbers in & 
sales district from only a single plant were to be continued. These were 
the questions in the minds of management which led to the linear pro- 
gramming study. y 

"The company was unique in that it had data for the cost of shipping 
from every plant to all 2500 jobbers. This information was readily avail- 
able. Much of the shipping is done by rail. Two different sizes of cars 
are used, The cost of shipping depends on the size of the car. The com- 
pany is not at liberty to use only the cheaper cars. It negotiates with 
the railroad for the percentage of each type of car to be used. These 
percentages are known, and they vary from jobber to jobber. 

The nature of the processing operations is such that the variable 
costs can be made directly proportional to the amount produced over & 
rather wide range. These variable production costs were known (with. 
moderate accuracy) for each of the twenty products. ‘The basic unit of 
measurement is what the company calls an “equivalent case.” The food 
products are actually packaged in a number of different sizes of containers, 
and hence some standard unit had to be used. Since the company in- 
ternally deals in equivalent cases, it was decided to use the equivalent 
ease as the fundamental unit in the analysis. The railroads base their 
charges on a weight basis, i.e., cost per 100 pounds. The traffic department 
kept the shipping costs in units of cost per 100 pounds. It was therefore 
necessary to convert these costs to an equivalent-case basis. 
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Each plant is set up to produce all twenty produets in roughly fixed 
proportions. The product mix can be varied somewhat at each plant. 
However, it was expected that there would be little variation in the 
product mix from plant to plant because the consumer demand was about 
the same in all sections of the country. Furthermore, the production costs 
are such that if one plant has favorable production costs for one product, 
it has favorable production costs for all products. This observation, to- 
gether with the fact that accounting costs and other internal problems 
increase considerably if a jobber receives different products from different 
plants, led to the restriction that all twenty products in the proper product 
mix must be shipped from one plant to a given jobber. 

The problem of proper assignment of sales distriets was studied first. 
Since there are only four plants, it is clear that an optimal solution will 
require that no more than four districts be supplied from two or more 
plants. "Thus practically every sales district will be supplied by one and 
only one plant. This fits in well with the established company policy of 
assigning each sales district to one plant. 

An examination of the sales data per sales district showed that the 
product mix shipped did not vary much from district to district. Conse- 
quently, it was possible to compute for each plant a variable production 
cost which was a weighted average of the cost for each of the products, 
The weighted average was based on the product mix which was known 
to apply to all sales districts. These produetion costs were therefore inde- 
pendent of the sales district to which the products were to be shipped. 

The sales-distriet problem was solved by using the sales figures of the 
preceding year to compute the demand for each district. This was done 
so that it could be determined whether an optimal shipping schedule had 
been used. The transportation method requires that there exist only one 
shipping cost to each sales district. In actuality, shipping costs vat 
within a sales district, depending on the jobber to which the products 
were shipped. To determine an average cost of shipping to & sales district 
from a particular plant, the demand of each jobber (in equivalent cases) 
was multiplied by the cost of shipment; these values were summed over 
all jobbers in the district and divided by the total cases shipped to the 
district. "This provided an average cost per case. The variable processing 
cost at plant 4 was added to the average transportation eost. of shipping 
one equivalent case from plant 7 to district j. This yielded the average 
total variable cost of producing one equivalent case of the proper product 
mix and shipping it to district j. Finally, the plant capacities (in 
equivalent eases) were determined for the given product. mix. 

The company owned a small digital computer, and the problem was 
solved on this computer by means of a transportation-method deck which 
was available for that computer. The computation time was one-half hour. 
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The same problem was also solved on the IBM 704 computer to test 
the program that was to be used for solving the larger problem. The 
computation time on the IBM 704 was four minutes. 

The optimal solution was quite interesting: About 10% of the seventy 
districts were assigned to a plant different from that of their current as- 
signment. The saving per year was about $150,000. In reality, the saving 
was a very small fraction of the total variable cost of about 25 million dol- 
lars per year. This shows that the traffic department had been quite suc- 
cessful in developing optimal shipping schedules. However, considerable 
savings, although a very small fraction of the total, were overlooked. 

Work on the jobber problem was begun next. For this problem, each 
jobber was considered to be a separate destination; the jobbers were no 
longer aggregated into sales distriets. Hence the problem had about 
2,500 destinations and 4 origins. A problem of this size can be solved 
only on the largest digital computers. Since the company computer was 
not large enough, it was decided to solve the problem on an IBM 704 
which has a memory capacity for 32,000 words. 

Although a good approximation was achieved by assuming that the 
product mix was essentially the same for the various sales districts, it 
was known that this assumption did not hold for individual jobbing points. 
In some cases, there were considerable variations in the product mix. It 
was decided that this initial study did not warrant the very considerable 
amount of work required by an analysis of the product mix at each of 
the 2,500 jobbing points and the computation of a variable production 
cost for each point. Instead, only a few jobbing points corresponding to 
the large cities were analyzed, and, according to their individual product 
mixes, a variable production cost was computed for each point. These 
few jobbers, however, represented a large fraction of the company’s total 
business. The production costs for the remaining jobbers were assumed 
to be the same as those used for the small problem. 

The company’s small digital computer was used to compute the total 
variable costs. The data were programmed in such a way that the small 
computer actually punched the cards (in proper format) that were to be 
used as the input data cards for the transportation program on the 
ІВМ 704. This reduced considerably the number of possible errors which 
may occur in punching the slightly over 2,500 cards needed for input data. 
To save time, the data were loaded into the IBM 704 f. rom tape; however, 
to put the information on tape, it must first be put on cards. 

The large problem was run off on the IBM 704 ; about two hours of 
machine time were required to reach an optimal solution. The results 
were quite interesting. Large additional savings were expected, but did 
not materialize. The optimal solution yielded a cost which was only 
about $50,000 below the optimal solution for the sales-district problem. 
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For this reason, management decided not to revise the sales districts 
radically. Instead, they chose to keep essentially the existing sales- 
district subdivision and use the optimal solution to the small problem 
as a basis for reallocation. Although the solution to the large problem 
did not lead to substantial additional savings, it was well worth the effort 
since it did point out that a revision of the sales districts would not result 
in any considerable cost reduction. 


12-5 Machine-assignment problems. The transportation method has 
been used with considerable suecess to solve certain types of machine- 
assignment problems. Consider a shop which produces a variety of differ- 
ent parts, usually to order. Assume that the shop is divided into several 
different departments. Each department contains a number of machines 
of the same general type, i.e., lathes, presses, or milling machines. How- 
ever, not all the machines in a department are identical; they may differ 
in age, size, or other characteristics, and hence be more suitable for making 
one type of produet than another. 

At the beginning of each week, the foreman of every department knows 
how much work is to be done in his department for the coming week 
(except for unexpected rush orders). He is faced with the task of assigning 
the work to the various machines in his department in such a way that 
production costs will be minimized. The cost of the operations on any 
one product depends on which machine does the work (it will be assumed 
that, in any department, none of the products need to be processed by 
more than one machine). 

We shall imagine that there are m machines (or machine types) and n 
products (or jobs). Let d;; be the time required to process one unit of 
product j on machine û, z;; the number of units of j produced on machine 7 
in the coming time period, a; the time available on machine i, b; the num- 
ber of units of j which must be completed, and cj; the cost of processing 
one unit of product j on machine i. The machine-assignment problem thus 
reduces to finding non-negative z;; which satisfy the constraints 
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"The machine-assignment problem formulated above has the form of the 
generalized transportation problem studied in Section 9-14. There we 
developed a generalized w»-algorithm which requires considerably fewer 
computations than would be needed in a direct application of the simplex 
method. Recall that unlike the standard transportation problem, optimal 
basic solutions to (12-5) need not have integral values. If the number of 
units to be produced is so small that a solution which does not yield inte- 
gral values of the ту is worthless, then some technique for solving linear 
programming problems in integers must be used. This requirement really 
converts the linear programming problem to a special type of nonlinear 
programming problem. We shall not attempt to discuss the solutions to 
such problems here. 

Under certain circumstances, the generalized transportation problem 
(12-5) ean be reduced to a standard transportation problem. It is always 
desirable to make such a reduction whenever possible, since it considerably 
simplifies the task of solving the problem. We shall consider one case 
in which (12-5) can be reduced to a transportation problem. 

Let us suppose that the products to be processed have the property 
that the ratio of the times required to process one unit of a given product 
on any two different machines is the same for all products. ‘Then select 
any machine, say, machine m, and call it the standard machine (to which 
all other machines will be compared). Thus, by the above assumption, 
dij/dmj = aj, and the same a; is obtained for every j. Now we can write 
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However, multiplying Уа; = b; by dwj, we obtain 
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where b; = dmjbj. Finally z can be written: 


г =} favis (12-7) 
m 

where fi; = ¢ij/dmj. Tt will be observed that aj is the number of standard 
machine hours available on machine i, bj is the quantity of product j 
which must be produced in the dimensions of standard machine hours, 
йу is the number of standard machine hours devoted to making produet j 
on machine т, and f;; is the cost per standard machine hour of processing 
product j. Equations (12-7a) through (12-70) represent a Standard 
transportation problem in the variables уу. To convert to the number 
of units of product j to be produced on machine ї, we use jj = Vij/dmj- 
Now we shall consider another case which turns out to be identical to 
the one just examined. Let us suppose that the ratio of times required 
to process one unit of any two different products on any given machine 
is the same for all machines, Then select any product, say, product n, 
and call it the standard product. Thus, by the above assumption, dij/din = 

Ву and is independent of i. Then, if we write 
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which is a standard transportation problem. Problem 12-13 asks the 
reader to provide a physical interpretation for the quantities introduced 
in (12-8). We leave it for the reader to show that if dij/dmj = cx is inde- 
pendent of j, then 4,у/4 = 8, is independent of i and vice versa, so that if 
the substitution (12-6) reduces a problem to a transportation problem 
the substitution (12-8) will also reduce it to a transportation problem, айа 
vice versa, 


12-6 Regular-time and overtime production. The problem of scheduling 
regular and overtime production can sometimes be solved with the help 
of the transportation method. Suppose that the manufacturer of a single 
product knows or can estimate what the demand for this product will 
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be for n future time periods. "These time periods may be days, weeks, or 
months. He wishes to schedule production over these next n time periods 
in such a way as to minimize the total variable costs. 

We shall assume that the product under consideration can be inven- 
toried, if necessary, for as many as n time periods. There will be a cost 
associated with keeping one unit of the production in inventory for one 
time period. 

In some circumstances, overtime production may be used to advantage 
and, in others, it ought to be avoided. For example, it might be known 
that if production were scheduled to follow exactly the demand pattern, 
considerable overtime would be needed in certain future time periods of 
especially heavy demand. On the other hand, certain quantities of the 
product could be produced on regular time in slack periods and inventoried 
until the demand exceeded regular-time production. The problem is to 
schedule production so that storage charges are balanced against over- 
time charges in such a way as to minimize the total variable cost. In 
another case, it may be advantageous to schedule overtime production 
in certain periods and build up inventory even when demand is low, be- 
cause the cost of production may be considerably lower in certain periods 
than in others, perhaps because raw-material prices show seasonal fluctu- 
ations, etc. 

We shall begin with a general formulation of the production-scheduling 
problem for a single product. Pertinent costs are: The cost c; of producing 
one unit in time period 7 on regular time, the cost d; of producing one unit 
in time period 7 on overtime, and the cost f; of storing one unit in inven- 
tory for time period 7. The production scheduler has to determine the 
schedule which minimizes the sum of production and storage costs. 

‘Let туу be the number of units of the product produced on regular time 
in period 7 for sale in period j, and yi; be the number of units produced 
on overtime in period 7 for sale in period j. We shall consider a; to be 
the number of units which can be produced in time period i on regular 
time, and а; the number of units which can be produced in time period 4 
on overtime (a; does not include the units which can be produced on regu- 
lar time). Finally, let b; be the number of units demanded in time period j. 
We wish to find the z;; and y;; which satisfy the production constraints, 
meet the demand requirements, and minimize the total variable cost. 

To formulate the problem mathematically, we begin by noting that the 
units produced in period 7 must be sold in period 7 or during a later period. 
Thus the production constraints take the form 


п 
D tij < а, i—1l...,m dD vi < а, $= 1 22/1 


j-i = 
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Any units sold in period j must have been produced in period j or during 
an earlier period. Thus the demand constraints take the form. 


j i 
D ry t+ DJ Yi = bh pass 5 (12-10) 
i=l i=l 
We wish to find z;; > 0, yi; > 0 which minimize the total variable cost 


non a л 
z= У Уса +) Ун 


i=l j=i i=l jai 


n j—l —1 
+> [> 2 (ais + ий. (12-11) 


j=2 i=1 (к= 


To see that this is the total variable cost, note that ZL; Citij is the cost 
of producing on regular time in period û. The total cost of producing on 
regular time is the sum over all time periods. The cost of storage for 
туу units made in period 7 on regular time and sold in period j is 
(fi + fia bs fjmi)mg. We assume that no storage charges are 
incurred during the selling period. To find the storage charges for all 
units sold in period j, we must sum (Уу /)(ж+у + yi) from û = 1 to 
i=j—41. Finally, the total storage charges are found by summing 
over all j. 

The problem described by (12-9) through (12-11) is a transportation 
problem. To see this more clearly, let us write 


nij = Yin а), 
н = a, b= 1.005% (12-12) 
and 
j—1 
cj = MIT i<j, ymo deest 
kei 
сМ, i»4 ism I= 1:200 b 
A (12-13) 
i di S fee doo entm a 1577045 
к= 


Quum М, >й Fa tena) 


We take M to be a large positive number so that the corresponding ij 
will be zero in an optimal solution. In this form we have a transportation 
problem with 2n origins and n + 1 destinations. 
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The fact that Eqs. (12-9) through (12-11) represent a transportation 
problem is more easily seen if we set up a tableau. In terms of the notation 
of (12-9) through (12-11), we obtain the tableau shown in Table 12-1. 
The last column gives the slack. 

Table 12-1 is interesting in that only about half the tableau is needed. 
None of the cells below the main diagonal need to be used. 

It is quite clear that the problem has no solution unless 


j E: 
Уа) > om NT 
k=l 


kel 


We shall assume that this condition is always satisfied. 

Note that it is very easy to determine the cost which is to be entered 
into each cell; it consists of the production cost plus the storage costs for 
the total number of periods during which the item is stored. 

Having formulated the problem in fairly general terms, we shall now 
concentrate our attention on an important special case. Assume that all 
costs are independent of the time period, i.e., c; = с, a= 4, = fi 


i=1,...,n. Then the function to be minimized becomes: 
noon nom n doo. 

2= с zu d У Уу ti D 2 O Dy + yu). (12-14) 
t=1 j=i i=l j=i j=2 i=l 


Suppose that the demand for each of the next ten time periods is repre- 
sented by the solid dots in Fig. 12-1, and the amount which can be pro- 


4 


Quantity 


Time 
Fieure 12-1 
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duced on regular time in each of the time periods by the open dots. For 
some of the time periods, the demand exceeds the regular-time capacity, 
"Therefore to meet. the demand, either the entire excess demand must be 
made on overtime (if this ean be done) or some units must be made in 
previous time periods and inventoried to help meet the excess demand, 

‘The nature of the minimum-cost schedule depends on the relative values 
of the difference between overtime and regular-time production, d — с, 
and the cost of storage per period for one unit /. When f is small com- 
pared to d — e, it will be cheaper to produce on regular time in periods 
when the demand is smaller than the regulur-time capacity in order to 
build up an inventory for the periods when demand is greater than regular- 
time eapacity, so that overtime work can be kept to a minimum. On the 
other hand, if f is roughly the sume as or greater than d — c, overtime will 
be used to produce items when demand exceeds supply, although these 


tion of regular-time production, overtime production, and storage charges, 
the combination yielding minimum costa. 
has been pointed out by Johnson [10], it is remarkably easy to find 
T solutions to the types of scheduling problems described above. 
No simplex iterations are required; an optimal solution can be obtained 
immediately without iterations. In fact, if we use the column-minima 
method for finding an initial basic feasible solution’ to a transportation 
problem, the resulting solution is optimal, provided we begin the technique 
with column 1, Le., with the first time period. The fact that the column- 
minima solution ix optimal is almost obvious. We ask the reader to prove 
‘it in Problem 12-8. 
general, production would not be scheduled in advance for some 
given number of time periods, say ten weeks, and then forgotten until 
the end of this period. The ten weeks would be used ax a planning horizon, 
and computations would be made each week to account for new orders, 
„ that is, each week a new production schedule would 
ten weeks, One schedules ten weeks in advance, 
plans for the first week will actually be used, because 
in the distant future can influence the production in the coming 
In other words, we may find that some items should perhaps be 
the coming week and be inventoried for delivery five weeks 
of thin procedure, future production can be adjusted 
numbers of units in inventory at the time of computation. 
technique deseribed above сап be generalized to the case where 
the same production facilities are used to make a number of different 
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ty be the time required to produce one unit of product k. Then b} = tar) 
is the total production time which must be devoted to the units of product 
k for which there is a demand in period j, Astume that a, hours are avail- 
able for regular-time production, and af hours for overtime production 
in period ti, Таке 24, to be the hours of regular-time production in period f 
devoted to the manufacture of produet k to meet the demands arising in 
period j, and y} be the corresponding overtime production. Finally, 
suppose that ca is the cost of producing one unit of product k on regular 
time, d, the cost of producing one unit of product k on overtime, and f, 
the storage cost for one unit of product k for one time period, 
We then wish to find non-negative zh, yf; which satisfy 


Sys $a, i94..." 


at (2-15) 
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and minimize 
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bel iwl md 


Here again we have a transportation problem, with the equivalent of 2n 
origins and mn + 1 destinations. The general structure of the tableau is 
shown in Fig. 12-2. The cells in the shaded areas are not used. As usual, 
we asume that set-up times and costa are negligible. 

In this multiproduct case, it is important that the total production time 
be а realistic measure of the availability of productive capacity. If the 
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Product 1 Product 2 Product m аск 


Capacities 


F10.. 12-2 Tableau structure for regular-time and overtime scheduling for а 
number of products. 


Юхлмрів: Consider a scheduling problem involving two products and 
two time periods, The costs, requirements, and the capacities on regular 
time and overtime are shown in Table 12-2, The cost of storage is in- 
cluded in the costs for products made in period 1 for sale in period 2. 

An initial solution is easily obtained by inspection (see Table 12-2). 
Since it was degenerate, one slack cell was added at a zero level to yield 
а basic solution. There is one cell with гу — cs > 0. The new tableau 
is shown in Table 12-3. The nonzero values of the variables have not 
changed. ‘Only the location of the zero-level variable has been changed. 


Taste 12-2 — TABLEAU 1 
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TABLE 12-3 — TABLEAU 2 
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profits. In addition, we wish to determine which process or combination 
of processes should be used to produce each product. 

To be concrete, let us suppose that the production of each product 
involves a sequence of operations on different machines. In general, a 
given machine can be used in the production of several different products. 
We shall assume that a number of different combinations of machines _ 
ean be used to produce a given product. For each way of producing a 


given product, we set down the sequence of machines used. Denote _ 


by 2; the amount of variable j produced per week; variable j refers to à, — 
given product produced by a given process. Since it is possible that _ 
several different processes can be used to produce a single product, several __ 
different 2; can refer to the same product. Let аг; be the time required _ 
on machine 7 to produce one unit of the product corresponding to variable | 
2; when the particular production process characterized by this variable _ 
is used. Let b; be the total time per week available on machine 7. Then, _ 
if there ате m machine types and set-up times are negligible, the z; > 0 | 
must satisfy the constraints 


Жар а Бал кут (12—18) 
j=1 


The number of variables, n, is the sum of the number of possible produc- 
tion processes for each item. 

In addition to the machine-time restrictions, there may be other con- 
straints, such as sales constraints. For example, it may be known that 
no more than a certain amount of each product can be sold, and that 
at least a certain amount of each product must be made to fill out the. 
line. These constraints take the form 

Xz; < d, (12-19) 


Xam. (12-20) 


The summations are taken over all variables which refer to the same 
product. 0 
Let cj be the profit made on the sale of one unit of variable j, that is, 
on the sale of one unit of a given product made by a particular process. 
We then wish to determine z; > 0 which satisfy the constraints (12-18) 


through (12-20) and which maximize the function 
n 
2= суду. (12-21) 
ici 


When we have solved this problem, we have determined the loading on 
each machine type, the total amount of each product to be produced, 
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and what fraction of each product should be produced by each of the 
possible processes. Clearly the solution to such a problem provides a 
great, deal of information. 


ExAMPLE: A small company makes three different products, 1, 2, 3. 
Each product requires work on two different machine types A, B. The 
shop has two varieties of type A machines whieh we shall designate 
by A1, A» and three varieties of type B machines, Bı, B2, Вз. Product 1 
can be made on any of the type A and type B machines. Product 2 can 
be made on any variety of type A machines, but must be processed on 
machines В; of type B. Finally, product З can be made orily on machines 
A, of type A and machines B of type B. 

The time in minutes required by one unit of each product on each 
variety of machine is given in Table 12-4, along with the total minutes 
of machine time available per week and the cost per week of running the 
machines at full capacity. The cost of raw materials for one unit of each 
product and its selling price are given in the last two rows of the table. 
We shall assume that the cost of running the machines is directly propor- 


TABLE 12-4 — Data FOR EXAMPLE 
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tional to the operating time and that this cost includes all other costs, 
aside from raw-materials cost (we shall ignore advertising costs, ete.). 

Product 1 can be produced in six different ways represented by the fol- 
lowing combinations of machines: (41, B3), (A1, B2), (Ai, Вз), (As, B), 
(A2, B2), (A2, Вз). Let z1, ..., ze denote the respective number of units 
of product 1 produced per week by each of these six processes. Product 2 
can be made in two ways represented by the combinations of machines 
(Ai, Bı), (A2, B1). Let z7 be the number of units of product 2 produced 
per week by process (A;, Bı), and let хв be the number produced by 
process (А 2, By). Product 3 can be produced in only one way, i.e., on 
the combination of machines (45, B). Take ту to be the units of prod- 
uct 3 produced per week. 

Next we shall formulate the machine-capacity constraints. There are 
five varieties of machines, and hence there will be five such constraints. A 
process of production, or an activity, is one way of producing one product. 
Each of the variables defined above refers to a different activity. There are 
nine activities in all. For activity j, let a;; be the time required for one unit 
on machine variety 2. Activity vector а; = [a1j, а2;; азу, алу, а5;) gives 
the times required on each variety of machines to produce one unit. Only 
two of the a;;-values will differ from zero. For example, activity 1 refers 
to producing product 1 on the (A;, B1) combination of machines. Thus 
ат = [5, 0, 6, 0, 0] since five minutes are required on machine variety А1 
and six minutes on machine variety В, to produce one unit of product 1. 
'The other activity vectors can be written down immediately. They are: 


az = [5, 0, 0, 4, 0], as = [5, 0,0, 0, 7], 
a, = (0,7, 6, 0, 0], as = (0, 7,0, 4, 0], 
as = [0, 7, 0, 0, 7], a; — [10, 0, 8, 0, 0], 
аз = [0, 9, 8, 0, 0], ag = [0, 12, 0, 11, 0]. 


The machine-capacity constraints then have the form Y; 9 za < b. 
Written out in detail, they are 


Br, 5u + 5ra - 10; < 6,000, 
Tes + 7х5 + 72g + 958 + 12x < 10,000, 

a + бу, + 812 + 8га < 4,000, 
Ar, + 425 + 11; < 7,000, 

Tag + 7x6 < 4,000. 


In addition to these, there may be some sales constraints, Suppose, 
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for example, that no more than 800 units per week of product 1 could be 
sold. This constraint would have the form 


1+ tg +23 + T4 +25 + xe < 800 


since the first six activities refer to the production of product 1. 

The final task is that of computing the unit profits to be used in the 
objective function. We shall illustrate the computation for activity 1, 
which describes the production of product 1 on machines (A1, B1). The 
cost on A, of producing one unit is (5/6000)(300) = 0.25. The cost on 
B, is (6/4000)(250) = 0.375. The raw material cost for one unit is 0.25. 
Thus the total variable cost of one unit is 0.875. The selling price is 1.25, 
so that the unit profit cı = 0.375. The unit profits for the other activities 
are computed in the same way. They are: с; = 0.300, c3 = 0.400, 
c4 = 0.400, cs = 0.325, ce = 0.425, су = 0.650, св = 0.861, со = 0.672. 
We wish to determine z; > 0 satisfying the constraints which maximize 


z = 0.3752, + 0.3002» + 0.40023 + 0.400z, + 0.32525 
+0.425x6 + 0.65027 + 0.861zg + 0.67210. 


We have shown how to formulate the problem. It will be noted that a 
rather small-scale problem can yield a rather large linear programming 
problem. If a large number of products can be made in many ways and 
requires numerous operations, the resulting linear programming problem 
can become quite large when sales restrictions and other constraints are 
included. ] 

The author assisted company personnel in solving one practical problem 
of the type considered in this section, which was concerned with the oper- 
ation of а textile mill. This particular mill made blankets. A number of 
blankets of different qualities and sizes were produced. Any given blanket 
required. operations on a number of different machines. Furthermore, 
certain blankets could be produced on more than a single combination 
of machines, i.e., by several different activities. In addition, there were 
maximum and minimum sales restrictions on some of the blankets. In- 
cluding the sales restrictions, the problem involved about 35 constraints 
and 50 variables. 

'The problem was solved by including first all sales restrictions, then 
only the maximum sales restrictions, and finally only the minimum sales 
restrictions. The company executives were quite interested in the results. 
However, the results were not too clear-cut, since considerable difficulties 
in obtaining data, especially costs and prices, made it impossible to ascer- 
tain just. how much operations could be improved. (For example, the 
company had a different selling price for almost every customer.) 
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12-8 Petroleum-refinery operations. In recent years, many petroleum 
companies have made efforts to optimize refinery operations by means 
of linear programming. In this section we shall describe briefly how such 
a problem can be attacked. The problem of optimizing refinery operations 
is extremely complicated, and we can do no more than present the barest 
outline of the procedure. 

The refining operation begins with the atmospheric distillation of crude 
petroleum. The distillation yields a number of different products (frac- 
tions), such as naphthas, straight-run oils, and bottoms. These fractions 
can be routed to a number of different units for further treatment or sent 
directly to blending of final products. For example, the straight-run oils 
may be sent to a eatalytie cracking unit to break down the long-chain 
hydrecarbons, and the naphthas may be sent to a catalytic reformer. 
A partial flow chart of a refinery operation is shown in Fig. 12-3. In 
any actual refinery there are many more fractions at each stage and also 
more stages. However, Fig. 12-3 will suffice for our purposes. 

Many final refinery products, such as gasolines, are formed by blending 
a number of different refinery streams. A product of given quality can be 
obtained from many different blends. There is a number of different 
decisions to be made in order to determine the final mix of refinery prod- 
ucts. First, a refinery often receives several different types of crude oil, 
and it must be decided which crude or mix of crudes should be run. Once 
this decision is made, the output of the crude-distillation unit is essentially 
determined. Next one must decide what fraction of the outputs of crude 
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distillation should go directly to blending and what fraction should go 
to successive conversion operations (cracking, etc.). Finally, at the 
blending stage, it has to be decided how to blend the various streams. 

In addition to determining the routing of the various streams, decisions 
have to be made on the operating conditions of a number of the units. 
It is possible to change the composition of the output of some units by 
changing some variables, such as temperature, pressure, and the so-called 
reflux ratio (parts of some of the outputs are recycled into the unit). 
We shall assume that the operating conditions for each unit are fixed. 
This means that once the input to the unit is specified, the various outputs 
are also determined. No attempt will be made to optimize the operating 
conditions of the various units. We shall concern ourselves only with 
the optimal allocations of the streams to the various units. 

The constraints fall naturally into several categories. First, there are 
the constraints representing the availability of the various crudes. Then 
there are the constraints on the equipment capacity, which limit the 
volumetric throughput in the various units. Next there are material- 
balance equations which relate the input into any one unit and the various 
output streams from the unit. Finally, there are the.constraints repre- 
senting the final products requirements. 

Let us return to the simplified, hypothetical refinery for which the flow 
chart is given in Fig. 12-3. We shall formulate the constraints by begin- 
ning with the crude and by following the various streams through the 
refinery. All variables will be assumed to be measured in the dimensions 
of barrels per day (bbl/day). A refinery may use ten or more different 
crudes. For simplicity, assume that our refinery uses two crudes which 
differ somewhat in chemical composition. Such differences can have an 
important influence on thé quantities and types of final products which 
can be made. Let the daily availabilities of these two crudes be 01, 02 
(these availabilities may change from week to week), and let ту, t2 be 
the bbl/day actually used. Then 


0 Kz <0, OS t2 < 02. (12-22) 


The crude-distillation unit will have a capacity limitation on the quantity 
of crude it can handle per day. If no more than A barrels per day of crude 
can be sent to crude distillation, then we must have 


Ксы е, (12-23) 


Now we are ready to study the output of the crude distillation unit. 
The naphtha stream of Fig. 12-3 will, in reality, be a number of dif- 

ferent streams. Suppose that our refinery produces light, medium, and 

heavy naphthas. One barrel of crude û (i = 1, 2) yields ai; barrels of 
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light naphthas, az; barrels of medium naphthas, and аз; barrels of heavy 
naphthas. Thus if yı, ys, уз are the barrels per day of light, medium, 
and heavy naphthas produced, we have 


Yı = ал121 F 01229, Уг = азіі + 42232, Уз = aiti + 3272. 
(12-24) 


Each naphtha stream is split into two streams: one goes to gasoline 
blending, and the other to reforming. Let y? be the bbl/day of naphtha, 
grade j, routed to blending, and y; the bbl/day routed to reforming. 
Thus е 

у= FY, ј= 1,2,3. (12-25) 


We can eliminate the у; and reduce the number of constraints by three 
if we substitute (12-25) into (12-24). This gives 


U +u — ала — ajra = 0, ј= 1,2,3. (12-26) 


Let us next consider the streams from crude distillation which are 
labeled oils in Fig. 12-3. Assume that there are only two such streams, 
light oil (oil 1) and heavy oil (oil 2). One barrel of crude i (i = 1,2) 
yields bı; barrels of light oil and bs; barrels of heavy oil. If ub, uj are the 
bbls/day of oil j (j = 1,2) which are routed to fuel-oil blending and 
cracking, respectively, then 


uj + — bazı — bjt = 0, | j= 1,2, (12-27) 


represents the material balances on the light and heavy oils. 

For the hypothetical refinery under consideration, the residuum stream 
of Fig. 12-3 is a single stream. One barrel of crude i (i = 1,2) yields 
d; barrels of residuum. If v, bbl/day of residuum. are routed directly to 
fuel-oil blending and vg to lube-oil processing, then the material balance 
on the residuum stream is: 


% + va — dizi — daz; = 0. (12-28) 


Material balances have now been written for all the streams emanating 
from crude distillation. We next turn our attention to the intermediate 
operations of reforming and cracking, The total daily input to the re- 
former is ут + 05 + y3. Let B be the maximum number of bbl/day 
of input which the reformer can handle, Then а 


Vi ys ys < B. (12-29) 


This takes care of the capacity constraint on the reformer. Next the 
reformer material balances must be formulated. 
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We shall imagine that only a single gasoline stream is produced by 
the reformer. One barrel of naphtha j sent to the reformer yields f; barrels 
of reformed gasoline. Thus, if w is the output in bbl/day of reformed 
gasoline, there is only a single material balance equation for the reformer, 
namely, 


w — fui — foya — favs = 0. (12-30) 


Consider now the cracking unit; this cracker yields only one cracked 
gasoline stream and one cracked oil stream. One barrel of oil j sent to 
the cracking unit produces g; barrels of cracked gasoline and Л; barrels 
of cracked oil. If r bbl/day of cracked gasoline and s bbl/day of cracked 
oil are produced, then the material-balance equations for cracked gasoline 
and oil are respectively 


r— giui — gous = 0, (12-31) 
s — hui — hous = 0. (12-32) 

If D is the capacity of the cracking unit, then it must be true that 
ujtus< р. : (12-33) 


We have now formulated all the constraints which apply to the cracking 
unit. 

For our hypothetical refinery, there is only one stream into the lube-oil 
processing unit. Assume that only one type of lube oil is produced, and 
that one barrel of residuum yields k barrels of lube oil. If q bbl/day of 
lube oil are produced, then the lube-oil material balance becomes 


q — kva = 0. (12-34) 

The capacity of the lube-oil processing unit is E bbl/day of input. 
Then the capacity constraint is 

va < Е. (12-35) 

There may or may not be restrictions on the quantity of lube-oil output. 


Suppose that the refinery can sell all the lube oil it produces, but that 
it must supply at least àbbl/day. This yields the additional constraint 


а> ô (12-36) 


At last we are ready to move to the final stage and write the constraints 
for the blending of the various final products. In practice, various quality 
restrictions are imposed on the final gasoline products. We shall simplify 
matters by assuming that for each gasoline, there is only a single octane 
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requirement to be satisfied. It is required that aviation, premium, and 
regular gasolines have octane numbers of at least ру, po, ps, respectively. 
Let m? (j — 1, 2,3) be the octane numbers of the three naphthas, m; the 
octane number of reformed gasoline, and m, the octane number of cracked 
gasoline. It will be assumed that the octane numbers of the various 
components blend linearly by volume. 

Denote by y}*, 4%”, y", j = 1, 2, 3, the bbl/day of naphtha j blended 
into aviation, premium, and regular gasolines, respectively. Similarly, 
"w^, w^, w" will denote the bbl/day of reformed gasoline, and ri; п 
will denote the bbl/day of eracked gasoline blended into aviation, pre- 
mium, and regular gasolines, respectively. The material balances for all 
components then read 


WW. —yP = 0, j-1428 (12-37) 
ш + w^ + w" — w= 0, (12-38) 
rr rT p= 0, (12-39) 


Equations (12-37) through (12-39) can be used to eliminate the yb шут 
in the previous constraints, thus eliminating the need for constraints 
(12-37) through (12-39). 

The total bbl/day of aviation fuel produced will be yt? yb + 
052 + w^ + r^. Thus, the octane constraint for aviation gasoline becomes:* 


bi 
mir + may" + туз + таш" + mart 


> pilvi + уба + ш" + r), (12-40) 
This сап be rearranged to read 
(i тт) уке + (py — my + (py — ту 
Tm = my)w* + (p, — m.) < 0. (12-41) 
Similarly, the octane constraints for premium and regular motor fuels are: 
(pa — тїї? -- (ps — my’? Æ (pa my 
+ (pa — mr)w” + (pa — mr" € 0; (12-42) 
(рз — my + (py — туу + (ps — mayi" 
+ (рз. — mr)w" + (рз — mF” < 0. (12-43) 


^ " If this inequality is not obvious to the reader, he will find its detailed deriva- 
tion in the following section, 
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j In addition to the octane requirements on gasoline, there may also be 

constraints on the quantities produced. A typical constraint, for example, 
— requires that the premium motor fuel be a given fraction Y of the output 
т. of regular motor fuel. This constraint has the form 


yv yb? + yk? س‎ Er? — yh Л yh! — тш тт 0. 
(12-44) 


We have yet to discuss the subject of fuel-oil blending. Suppose that 
the refinery produces one jet fuel and one other fuel oil. Let s^, s^ be 
the bbl/day of cracked oil, u, u?” the bbl/day of light oil, uf, ul’ the 
bbl/day of heavy oil, and vf, of the bbl/day of residuum blended for 
jet fuel and fuel oil, respectively. We then have the following material- 
balance equations: 


s! d. sf — 8 0, (12-45) 
vital а L0 ue tuk = 0, (12-40) 
he i+ — ть = 0. (12-47) 


These equations could, of course, be used to eliminate и}, uh, 8, v, in the 
preceding constraints. 
A limit on the vapor pressure is the only quality constraint we shall 
impose on the jet fuel. The upper limit to the vapor pressure is to be ¢. 
If qı, q2, qe, Gr are the vapor pressures of the light, heavy, cracked, and 
; residuum oils, respectively, then the vapor pressure constraint can be 
f written 3 у 
qut e әм + qui! + awh S olut + ul + а + vt), 
or (12-48) 


(à — qul + (6 о) ш + — qe! + — qh > 0, 


provided we assume that the vapor pressures of the constituents blend 

"linearly by volume. In addition to this constraint, constraints on the 

boiling range and aromatic content of the jet fuel are often imposed in 

practice. k 

No particular quality specifications will be given for the fuel oil. How- 

ever, we require that the various components be blended in fixed propor- 

M tions. Expressing the quantity of all other components as à certain 
fraction of the light oil used, we obtain the following set of constraints: 

ull — youl! = зра vb puy = 0. (12-49) 


1 ü 
In addition, there may be maximum and minimum limitations on the 
output of fuel oil. 
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We have finally completed the task of formulating the constraints for 
our oversimplified, hypothetical refinery. All the variables must, of course, 
be non-negative. Although many simplifications were introduced, the 
number of constraints is considerable. Any attempt to represent realis- 
tically the operations of an actual refinery can easily lead to 150 or 200 
constraints, and thus to a very large linear programming problem. It 
should be noted that one must be careful in imposing quantity constraints 
on the final products, since their output cannot be arbitrarily specified. 
For example, it is not possible to produce only fuel oil and no gasolines, 
or vice versa. This fact has presented some problems for the petroleum 
industry. For example: To produce enough fuel oil to meet the winter’s 
demand, it is sometimes necessary to produce more gasoline than is 
really needed; this, of course, leads to excessively large stocks of gasolines, 

We have not yet discussed the objective function we wish to maximize. 
Tf arbitrary quantities of any final product could be sold, then no upper 
limits would have to be placed on the output of any product, and we 
would maximize the function 


Z = cui" + y + y + w® + т“) + es? + yb? + yh? + ш” +7”) 
Helyi + yx + ys + wl + r") + eju + ul в? + vl) 
+ eu! + ul + s + vf) + сы, (12-50) 


where Ca, Cp, Cr, су, Cf, Cu are the profits per barrel of aviation, premium, 
and regular gasolines, jet fuel, fuel oil, and lube oils, respectively. 

However, as has been noted above, the demand for one product (such 
as fuel oil) may require that another product be produced in a larger 
quantity than can be sold immediately. Hence, profits may be maximized 
by minimizing the quantity of gasoline produced in the course of processing. 
a given amount of fuel oil. Or, if gasoline is in demand, we may wish to 
minimize the production of fuel oil for a given gasoline-production rate. 
Thus we see that it is not always simple to decide what the objective 
function should be. 

Alan Manne [12] has made an interesting study of the refining in- 
dustry as a whole, along the lines presented in this section. He wished 
to determine the maximum jet-fuel production on the basis of a given 
output quota for certain other products. 


12-9 Blending problems. We shall now discuss the application of 
linear programming to so-called blending problems. In the preceding 
section, we encountered some excellent examples of blending problems. 
We noted that many of the final refinery produets were formed by blend- 
ing a number of different refinery streams. In general, blending problems 
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refer to situations where a number of components are mixed together to 
yield one or more products. Usually, there are restrictions on the available 
quantities of raw materials, restrictions on the quality of the products, 
and perhaps restrictions on the quantities of the products to be produced. 
However, there are usually infinitely many different, ways in which the 
raw materials can be blended to form the final products while satisfying 
the various constraints. It is desired to carry out the blending operation 
so that some given objective function will be optimized. 

The petroleum industry does not provide the only example of blending 
operations. In the manufacture of paints, and even in the manufacture 
of ice cream, blending problems are encountered. Similarly, blending 
problems exist in the steel industry. As an example, consider the determi- 
nation of the charge for a blast furnace. Several varieties of ore, along with 
scrap steel, limestone, and coke, are mixed together to form the charge. 
Many blends are possible. The optimal blend depends on what the ob- 
jectives are. 

Linear programming forms a natural technique for optimizing blending 
operations provided that the various quantities of interest blend linearly. 
It is by no means true that all physical quantities blend linearly, and hence 
if linear programming is applied in a case of nonlinear blending, it will 
yield only an approximate answer. In many instances, however, the 
assumption of linearity has given answers which were good enough to 
make linear programming a very useful tool for studying blending 
operations. 

We shall begin with a discussion of the blending of motor gasolines. 
Gasoline blending is a very important problem because a small percentage 
improvement in operations could mean millions of dollars to the oil com- 
panies. Many of the larger oil companies have been using linear program- 
ming for some time to solve their blending problems. Some have claimed 
that substantial amounts of money were saved by means of linear pro- 
gramming. Gasoline-blending problems alone, even if divorced from the 
operations of the refinery as a whole, can lead to quite large linear pro- 
gramming problems. One oil company has worked with a gasoline-blending 
problem involving eighty constraints. It required more than two hours 
of time on the IBM 704 to reach an optimal solution. 

Any motor fuel must satisfy a number of quality requirements. Im- 
purities, such as sulfur, must not exceed а given concentration, the vapor 
pressure (referred to in the industry as the Reid vapor pressure) must 
be within a certain range, and the octane number must equal or exceed 
some minimum value. 

Most of these requirements are met by measures which are independent 
of the blending operation. For example, some impurities are controlled 
by the proper selection of crudes, while other impurities are eliminated 
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by chemical treatment. The vapor pressure of the motor fuel is often 
adjusted by adding butane to the fuel. The raw refinery stocks are passed 
through a debutanizer to reduce their vapor pressure for easy storage. 
The butane is added after the fuels have been blended. In the winter, 
more butane is added than during the summer, so that ignition will be 
rapid in cold weather. Tetraethyl lead is usually also added to motor 
fuels. Since this is a very dangerous substance, it is not added to the 
blended fuels until they are ready to be shipped. The only quality re- 
striction which remains to be taken care of in the blending operation is 
the octane number.* 

The quality requirement on the octane number refers to more than а 
single restriction. Gasoline is a mixture of a number of hydrocarbons. 
When gasoline is heated, it will begin to vaporize. Initially, the vapor 
will contain the lighter hydrocarbons (those with a smaller number of 
carbon atoms). With continued heating, the composition of the vapor 
and liquid changes; furthermore, the boiling point of the liquid increases 
as it becomes more concentrated in the heavier hydrocarbons. Thus, 
specification of the boiling point specifies the liquid composition. 

For any one gasoline, the minimum octane number is usually specified 
for several boiling ranges. Although the refineries attempt to obtain one 
octane number for all boiling ranges, this cannot be done exactly. A 
single octane number is desirable for the following reasons: When the 
mixture of gas and air is inserted into the cylinder of an engine, it is 
partially vaporized. Unfortunately, the composition of the liquid and 
vapor may vary from cylinder to cylinder. If the octane number of the 
liquid-vapor mixture in the various cylinders varies considerably, the 
tendency of the engine to knock is aggravated. 

Let us turn to the details of the blending problem. Assume that three 
refinery streams 1, 2, 3 are blended to form regular, r, and premium, 7, 
motor fuel. The availabilities of the refinery streams 1, 2, 3 are known to 
be Ay, Ao, Аз bbl/day, respectively. Suppose the minimum octane 
numbers of regular. and premium gas are to be б, 7, respectively, in the 
250-300°F boiling range. The corresponding octane numbers in the 
350-400°F boiling range are to be à», Уз. Let €j; be the octane number 
of refinery stream j in the 250-300°F boiling range, and €;2 be the octane 
number of stream j in the 350-400°F boiling range. 

We shall imagine that the refinery under consideration is a small Gulf 
Coast refinery which produces only premium and regular motor fuels— 


__* The quantity of tetraethyl lead added to motor fuels has an important bear- 
ing on the antiknock performance of the final fuel, and sometimes it is desirable 
to include it as one of the components to be blended. We shall assume that the 
quantity of tetraethyl lead to be added is fixed; and we shall not consider it in 
the blending problem. 
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no aviation gasoline. However, it can sell the refinery streams at prices 
of сү, сә, сз per barrel, respectively. The selling prices (tank-wagon price) 
of premium and regular gasolines are ср and cr, respectively. The refinery 
is under contract to blend at least R bbl/day of regular gasoline. The 
amount of premium gasoline is always a fixed fraction, f, of the amount of 
regular gasoline. The refinery can sell all regular and premium gasolines 
it produces. We wish to determine how the raw stocks should be blended 
to form the motor fuels and, in addition, how much of each motor fuel 
ought to be made in order to maximize profits. 

Denote by zy; > 0, = 1, 2, 3 the bbl/day of raw stock j used to 
blend premium gasoline, and by 2,; > 0, j = 1, 2, 3 the bbl/day of 
raw stock j used to blend regular gasoline. The constraints on the avail- 
abilities of the raw stocks read 


туу + tri S А» j= 1,2, 3. (12-51) 
At least R bbl/day of regular gasoline must be produced. Therefore 
та + Tra + arg > R. (12-52) 


To write (12-52), we must assume that the volumes are strictly additive. 
This is a very good approximation. The amount of premium fuel produced 
must be a fixed fraction of the regular fuel. Thus 


уу + хра + pa — fin — Jara — {йз = 0. (12-53) 


We now come to the octane constraints. In reality, octane numbers 
do not blend linearly. This is especially true when aromatic hydrocarbons 
are added. However, over limited regions, the linearity assumption ів а 
good approximation. According to this hypothesis, each constituent con- 
tributes to the octane number of the mixture an amount equal to the 
product of its octane number and its fraction of the total volume of the 
mixture; for example, the volume fraction of j in regular gasoline is 
#ry/ (te1 + eg + 2ra). Thus, at 250-300°F the octane constraint for 
regular gas is: > 


t, 
"LI rz + шкы кен m aa Fag 2 9 
or 
eza + engra + ёз1лүз > 1n + 7,2 + Tra), (12-54) 
or 


! 
(ду — €11)te1 + (81 — в) + (81 — 6327 < 0. 
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The remaining octane-number constraints are: 


(ry — єп) + (Y1 — €21)252 + (Yı — єзү)льз < 0, (12-55) 
< 


0, (12-56) 


(82 — €12)tr1 + (52 — Єз), + (82 — €32)t73 


(Ya — €i)tpr + (Үз — €22)t52 + (Ya — єзз)хьз X 0. (12-57) 


The constraints have been formulated. Now we shall discuss the ob- 
jective function. Since we assume that a fixed amount of each raw stock 
is produced each day, and since the entire output is sold, the costs are 
fixed and do not enter into the optimization. Thus the function to be 
optimized is 


2 = Cy(Zp1 + 2р2 + tps) + ск + tre + тз) + €1(A — tpi — ха) 
+ ¢2(Ao — apa — туз) + сз(Аз — tpa — Tra). 
Equivalently, we wish to. maximize 
2 = (Cp — €1)tpi + (Cp — €2)tpa + (Cp — сз) тъз + (Cr — сі) 
+ (с, — e2)m.2 + (e, — ca)r,s. (12-58) 


The solution to the blending problem not only tells us how the raw 
Stocks should be blended, but in addition, it indicates how much of each 
gasoline will be blended, and therefore what quantities of raw stocks will 
be sold directly. It may amuse the reader to know that the refineries do 
not completely trust these blending computations. They will often test 
the gasolines in octane engines to see whether they do indeed satisfy the 
octane requirements. 

Diet problems form another interesting class of blending problems. 
Dieticians tell us that a balanced diet must contain certain quantities of 
nutrients, such as calories, minerals, and vitamins. Suppose that we are 
asked to determine from a given number of foods the lowest-cóst diet 
which satisfies the minimum requirements for a balanced diet. Assume 
that n foods are available, and that there are m nutrients which, in given 
minimum amounts, must be supplied by the diet. 

Let ai; be the number of units of nutrient i in one unit of food j. 
Furthermore, imagine that xj units of food j are to be used per day, and 
that the diet must supply at least b; units of nutrient i per day. Denote 
by c; the cost of one unit of food j. We shall assume that if z; units of 
food j are eaten, a;;z; units of nutrient i are provided. In addition, we 
imagine that the amount of nutrient 7 obtained by eating several foods 
is the sum of the amounts of nutrient + in each of the foods eaten. Then 
the: ту > 0,j:— 1,...,n which provide a minimum-cost. diet are an 
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optimal solution to the linear programming problem 


s n 
Dd az; > bs, Xm mug 025. (12-59) 


j=t ј=1 


It should be noted that the diet problem, as formulated, does not 
provide for any variations in the diet. The same mixture must be eaten 
day in and day out. Furthermore, it does not take into consideration 
whether the diet is at all palatable. Finally, there are no constraints 
that place an upper limit on the quantity of any nutrient provided by the 
diet. It may be desirable to include upper limits on the nutrients so that 
the consumer of the diet would not get too many carbohydrates, for 
example. 

As indicated in Chapter 1, Stigler was probably the first to find a 
minimum-cost diet. He did not use linear programming. Later Dantzig 
and Laderman solved the same problem by the use of linear programming. 
Needless to say, the diet was not one that anybody would care to eat for 
any length of time. However, it showed that in terms of 1939 prices, a 
person could have a balanced diet for the remarkably small sum of about 
$40 per year. 

Minimum-cost diets have been applied in practice by farmers and 
ranchers for feeding various kinds of livestock. Other potential fields of 
application are space exploration and nuclear submarines, where the goal 
is to minimize the weight of food needed for an adequate diet rather than 
to minimize cost. 
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PROBLEMS 


12-1. A dog-food manufacturer has three plants which ship to six major dis 
tribution points. The tableau below contains the shipping costs per case am 
the plant capacities and predicted demands for the coming year in millions 
cases. The variable production costs per case at the plants are: (1) 0.85, (2) 0.94, 
(3) 0.72. Determine the minimum cost production-shipping schedule. 


12-2. Prove that if the variable production costs are the same at every plant, 
one can obtain an optimal production-shipping schedule, using transportation 
costs only, > 

12-3. Prove that if every plant must be operated at capacity, then one can 
obtain an optimal shipping schedule, using transportation cost only, even though 
variable production costs differ from plant to plant. 4 

12-4. A farmer has 100 acres which can be used for growing wheat or corn. 
The yield is 60 bushels per acre per year of wheat or 95 bushels of corn. Any 
fraction of the 100 acres ean be devoted to growing wheat or corn. Labor re- 
quirements are 4 hours per acre per year, plus 0.15 hour per bushel of wheat 
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and 0.70 hour per bushel of corn. Cost of seed, fertilizer, ete., is 20 cents per 
bushel of wheat and 12 cents per bushel of corn. Wheat can be sold for $1.75 
per bushel, and corn for $0.95 per bushel. Wheat can be bought for $2.50 per 
bushel, and corn for $1.50 per bushel. 

In addition, the farmer may raise pigs and/or poultry. The farmer sells the 
pigs or poultry when they reach the age of one year. A pig sells for $40. He 
measures the poultry in terms of one “pig equivalent” (the number of chickens 
needed to bring in $40 at the time of sale). One pig requires 25 bushels of wheat 
or 20 bushels of corn, plus 25 hours of labor and 25 square feet of floor space. 
An equivalent amount of poultry requires 25 bushels of corn or 10 bushels of 
wheat, 40 hours of labor, and 15 square feet of floor space. 

The farmer has 10,000 square feet of floor space. He has available per year 
2,000 hours of his own time and another 2,000 hours from his family. He can 
hire labor at $1.50 per hour. However, for each hour of hired labor, 0.15 hour of 
the farmer's time is required for supervision. How much land should be devoted 
to corn and how much to wheat, and in addition, how many pigs and/or poultry 
should be raised to maximize the farmer's profits. 

12-5. A caterer has undertaken a contract for a series of dinners to be given 
by an exclusive New York Club. There will be n dinners, one on each of n suc- 
cessive days. The caterer must purchase napkins especially for these dinners 
because the Club has requested a special type of napkin which has not been used 
before and will not be used again. On day k, k = 1,..., n, а dinners will be 
served, and hence a, napkins will be needed. Two types of laundry service are 
available to the caterer. Regular service takes p days (if the laundry is sent at 
the end of day k, it can be used again on day k + p), and the costs are 8 cents 
per napkin. A faster service takes q < p days and costs Y > 8 cents per napkin. 
New napkins cost æ cents each, The caterer would like to minimize the costs 
associated with purchasing and laundering napkins, The problem of deciding 
how many napkins to buy, and of determining the number of napkins to be sent 
to the regular and to the fast laundry service each day is to be formulated as a 
linear programming problem. Show that the problem, called the caterer problem, 
can be solved by the transportation method (Jacobs, W. W., “The Caterer 
Problem,” Nav, Res. Log. Quart. 1, 1954), Hint: Let z; be the number of napkins 
used on day ї which have never been laundered before. Let yy be the number of 
napkins soiled on day i which are sent to the regular laundry service to be used 
again on day j. Take t to be the number of soiled napkins not sent to the laundry 
on day i. There are no costs associated with maintaining inventories of laundersd 
or soiled napkins. What will be the value of yy when j < i+ р? Will it ever 
be profitable not to send some soiled napkins to the laundry on day +, but to 
send them later on, say, day j? Sketch the structure of the transportation-ty pe 
tableau, 

12-6. Solve Problem 12-5 for the special case where a = 100, a2 = 200, 
45 = 300, a4 = 400, as = 200, ag = 100, a = 1,00, 8 = 0.05, Y = 0.10, 
р =4,@0 = 2 Р ; 

12-7. Generalize the regular-time versus overtime production-scheduling 
problem discussed in Section 12-6 to the case where an initial inventory of a 
units of the product is available and where, in addition, the final inventory at 
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the end of the nth time period is to be 8. Hint: Add a row to represent the 
initial inventory alloeated to the requirement of each time period. 

12-8. Prove that for the scheduling problem of a single product discussed in 
Section 12-6, the column-minima technique for determining an initial basic 
feasible solution yields an optimal solution. Hint: Let æ be the difference between 
the costs in rows and s, r > s, in column К. What is the difference between the 
costs in rows r and s in column q, q > k? 

12-9. Show that for production-scheduling problems of the type discussed 
in Section 12-6, the column-minima solution is optimal even if the manufac- 
turing costs are nonlinear, provided e;(z") > с(т') if z” > x’ [ci(z) is the cost 
of producing z units in time period i]. 


f 
4 


Ficure 12-4 


12-10. Consider the following nonlinear programming problem: Determine 
an x > 0 which minimizes z = 57. | су(ту) and satisfies the constraints Ax = b. 
Show that, to an arbitrary degree of accuracy, this problem can be reduced to à 
linear programming problem, provided that c;(z,), j = 1,..., n, is а continuous 
function of zj, d?c;/dz? > 0, and c((0) = 0. Hint: Note that each c; is a funo- 
tion of z; only. By assumption, each ¢-curve is similar to that shown in 
Fig. 12-4. Use the points aij, азу, азр... to divide the z-axis into a set of in- 
tervals. Then define a set of variables aj such that 0 € Zij € aij — 0-1, 
with the additional provision that туу = 0 unless all z are at their upper bounds 
wheni < k. With these assumptions, z; = Улуу. Now in theinterval aj—1,; < 
Zi < aij write cí(zj) = Yaz; Yia, where Yir, Уса are chosen so that cj(ai—1,/) = 
Ya, clasi) = Ya(ai — a1) + Yia. Set up the programming problem in terms 
of the variables ту. Show that it is а linear programming problem. Also show 
that it follows automatically that an optimal solution has the property ту = 0 
unless all z;; are at their upper bounds when i < k. Why is it necessary to re- 
quire that с;(0) = 0 for all j? 
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12-11. For the coming month, a domestic airline is faced with the problem 
of allocating available aircraft to its various routes. "There are n different routes 
in all, and the expected demand in numbers of passengers wishing to make a 
one-way trip over route j will be denoted by b. To simplify the problem we shall 
suppose that if the plane makes one or more intermediate stops before the final 
destination, and if someone gets off at an intermediate stop, this person is re- 
placed by another passenger boarding the plane, "Thus the demand represents 
one-way trips from the origin to the final destination. The airline has m different. 
types of planes. A plane of type 7 flying route j сап carry dy passengers. Some 
planes may not fly certain routes and are not to be considered for allocation to 
these routes. If a plane of type 1 is used, the profit per passenger on route j 
will be taken to be pij. Assume that а; planes of type i will be available for 
service in the coming month. We wish to determine the allocation of planes to 
routes which will maximize profits for the coming month. Set up the problem 
as n linear programming problem. To set up the model, what implicit assumptions 
have to be made about the nature of the demand? 

12-12. Generalize the model developed in the previous problem to include 
first-class and tourist flights. Assume that the expected demands on the various 
routes for both first-class and tourist flights are given. Suppose that a given 
flight will not mix first-class and tourist passengers. 

12-13. Provide a physical interpretation for each of the variables introduced 
in Eq. (12-8). 

12-14. For the production-transportation problem discussed in Section 12-3, 
what problems are created by inventories carried at the manufacturers and job- 
bers? How can inventories be accounted for in the model? 

12-15. A small Gulf Coast refinery blends five raw stocks to produce two 
grades of motor fuel, A and B. The number of barrels per day (bbl/day) of 
each raw stock available, the octane numbers and the cost per barrel are given 
in the following table: 


The octane number of motor fuel A must be at least 95, and that of motor fuel B 
at least 85. Assume that a contract requires that at least 8000 bbl/day of motor 
fuel B must be blended. However, the refinery can sell its entire output of fuels 
A and B. Motor fuel A is sold to distributors at $3.75 per bbl and motor fuel 
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B at $2.85 per bbl. All raw stocks not blended into motor fuel with an octane 
number of 90 or more are sold for use in aviation gasolines at $2.75 per bbl, 
and those of octane number 85 or less are sold at $1.25 per bbl for use in fuel 
oils. In order to maximize daily profits, how much of each motor fuel should be 
made and how should the raw stocks be blended? Is it really necessary to know 
the cost per barrel of each stream? 

12-16. Discuss the problems of obtaining data for any practical application 
of linear programming. Consider the effects of errors in estimating various 
parameters and the costs of obtaining the data. 

12-17. For the coming month, the foreman of the lathe department in a small 
machine shop must schedule six jobs on five lathes. The pertinent data are 
provided in Table 12-5. No job requires time on more than one machine. The 
costs shown include raw material costs. Set up the problem as a linear pro- 
gramming problem and solve it. 

12-18. In machine-loading problems, it sometimes turns out that tooling up 
for à job is a rather costly operation, and hence it is not desirable to split jobs 
(i.e., have more than a single machine work on a job) if this can be avoided. 
Develop an approximate method for assigning jobs to machines for the case 
where jobs are not split, in order to minimize costs. Try to find such a solution 
to Problem 12-17. How does the cost of your solution compare with that for 
the optimal solution to Problem 12-17, where splitting is allowed? Use the 
costs given in Problem 12-17. Hint: For the general procedure compute the 
total cost, including setup costs for doing each job on every machine where it 
can be processed. Then compute for each job and each machine the difference 
between the cost on the given machine and the machine that does the job at 
lowest cost. Enter these numbers in a transportation-type tableau along with 
the times required to complete the entire job. First assign jobs to machines 
which can do the jobs at least cost; then transfer jobs from oyerloaded machines, 

12-19. In practice, when machine loading must be done once a week or more 
often, it may be difficult to solve exactly the suitable linear programming prob- 
lem, especially in a small company which does not озуп а computer. Try to 
develop some approximate procedures for solving machine-loading problems, 
which require. very little computation and which can be carried out quickly. 
Hint: See preceding problem. Д 

12-20. Solve the example in Section 12-7. In solving the problem, ignore 
the sales constraint on product 1. How does the optimal solution change when 
this sales constraint is included? It is expected that in the near future competi- 
tion will force the selling price of product 3 down to $2.30. How will the optimal 
solution (including the sales constraint) change if this happens? How much can 
the selling price of product 3 be decreased without changing the optimality of 
the original solution? ; d 

12-21. A manufacturer of metal office equipment makes desks, chairs, cabi- 
nets, and bookcases. The work is carried out in three departments: (1) metal 
stamping, (2) assembly, (3) finishing. The pertinent data are presented in the 
table below. ‘The times are printed in boldface type, while the cost incurred for 
one unit in each department is in lower-case Roman type. 
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Hours per 
Department Desk Chair Cabinet | Bookcase week 
available 
. 3 1.5 2 2 
Stamping 15 8 12 12 800 
10 6 8 7 
Assembly 30 18 24 21 1200 
Seid 10 8 8 7 
Finishing 35 28 25 21 800 
Selling price 175 95 145 130 


Finishing work can also be subcontracted to an outside firm at a 30% increase in 
the cost of finishing. What should the rate of production of each of the items be 
in order to maximize weekly profits if no restrictions are placed on the number 
of any item to be made. What is the optimal product mix if one chair is to be 
made for each desk? How much could weekly profits be increased by a 10% 
increase in the capacity of the finishing department? What is the optimal 
product mix if it is necessary to make at least one cabinet and one bookcase for 
every three desks produced (assume that one chair is made for each desk)? 

12-22. Consider a firm which manufactures m different products, using com- 
mon productive facilities. Each product may, for example, require operations on 
several different machines. Assume that there is a number of different ways to 
make each product. Furthermore, assume that each product can be inventoried. 
An attempt is being made to schedule production for n future time periods. 
The demands for the individual products vary widely from period to period. 
Suppose that the demand for each product can be forecast for each of the coming 
periods. The selling prices of the products may fluctuate from period to period, 
25 may thé costs of raw materials. Assume that the cost of carrying each item in 
stock for one period is known. Set up the linear programming problem which 
will determine how much of each product should be made in each period and 
which activities should be used to produce them. Sketch the structure of the 
matrix. What might be an efficient procedure for solving such a problem? Can 
you provide an example of one or more industries where’ such a problem has 
practical significance? 

12-23. A food processor has m farms, n plants, and k jobbers. Let us consider 
one product, say, tomato soup. Tomatoes can be grown at all m farms; how- 
ever, the cost of growing them varies from farm to farm. Take f; to be the cost 
per bushel at farm i. Any farm can ship to any plant, and any plant may ship 
to any jobber. Let cij be the cost of shipping one bushel of tomatoes from farm 
i to plant j, and let dj; be the cost of shipping one case of tomato soup from 
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plant j to jobber k. Assume that 8 bushels of tomatoes make one case of soup. 
The variable cost of making one case of soup at plant j is pj, and the capacity 
of the plant for tomato soup is T; cases. Take b, to be the number of cases of 
tomato soup required at jobber k for the coming year and let a; be the maximum 
number of bushels of tomatoes which can be grown at farm i. Set up the linear 
programming problem whose solution minimizes the cost of growing, processing, 
and transportation, and which meets the jobber requirements for tomato soup. 
Can you devise an efficient computational scheme for solving this problem? 
Can you list other industries where a problem of this type is relevant? 

12-24. Discuss how the decomposition principle could be used to solve prob- 
lems of the type illustrated in the example of Section 12-7. Under what cir- 
cumstances would the example of Section 12-7 be a generalized transportation 
problem? 

12-25. Set down the structure of the constraints for the gasoline-blending 
problem diseussed in Section 12-9. How could the decomposition principle be 
used to solve such problems? Under what circumstances would a gasoline- 
blending problem be reducible to a generalized transportation problem? 

12-26. Formulate the regular-time and overtime production scheduling prob- 
lem of Section 12-6 in a different way, using as variables the total quantities 
produced in any period on regular and overtime and the inventories at the be- 
ginning of each period. In terms of these variables, is it so obvious that the 
problem is a transportation problem? 

12-27. A speculator operates a large silo for storing corn. In the real world, 
he will not know what the selling price or the purchase price of corn will be in 
future periods. His success will depend on how well he can predict these values. 
Instead of this real-world situation, consider the simple case where he knows 
(or can predict very accurately) the prices for which he can buy and sell corn 
in each of n future time periods. Let R be the capacity of the silo in bushels and 
let d; be the cost per bushel in period ї and f; the selling price per bushel in period 
i. Assume that anything bought in period 7 cannot be sold until period i+ 1 
or a later period, i.e., assume that corn purchased in period 1 does not reach the 
silo until the end of period i. Take ту, y; to be the quantities bought and sold in 
period i. Formulate as a linear programming problem the problem of determin- 
ing how much to buy and sell in each of the coming n periods so as to maximize 
profits. Assume that at the beginning of the first period K bushels of corn are 
in the silo. Also assume that there are no inventory-carrying charges. Show that 
an optimal solution will have z; = 0, or R — K, or Randthaty; = 0, or R, or K 
for all i. Devise a simple way of solving the problem which does not involve 
linear programming. This problem is often referred to as the warehouse problem. 

12-28. Solve the warehouse problem formulated in Problem 12-27 for 10 
time periods during which, beginning with the first period, the selling prices per 
bushel are 2.10, 2.50, 2.75, 2.80, 2.85, 2.65, 2,55, 2.30, 2.20, 2.40, and costs 
per bushel are 2.45, 2.50, 2.35, 2.20, 2.00, 1.80, 2.00, 2.20, 2.30, 2.40. The silo 
capacity is 100,000 bushels, and at the beginning of the first period 40,000 bushels 
are in the silo. 

12-29. Would the nature of the optimal solution to Problem 12-27 be changed 
if inventory-carrying charges were added? 
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12-30. Generalize the situation discussed in Problem 12-27 to a warehouse 
which can stock n different items. Formulate as a linear programming prob- 
lem the question of determining how much to buy and sell of each item each 
period in order to maximize profits. What can be said about the nature of an 
optimal solution? This problem is discussed in A, Charnes and W. W. Cooper, 
“Generalizations of the Warehouse Problem,” Operational Research Quart., 6, 
1955, 

12-31. Consider the refinery problem formulated in Section 12-8. Assume 
that the daily availabilities of crudes 1 and 2 are 20,000 and 30,000 bbl/day, 
respectively. The capacity limitation on crude distillation is 45,000 bbl/day. 
One barrel of crude 1 yields 0.10, 0.20, 0.20 bbl of light, medium, and heavy 
naphthas, and crude 2 yields 0.15, 0.25, 0.18 bbl of the same naphthas. One barrel 
of crude 1 yields 0.12 and 0.20 bbl of light and heavy oils, respectively, and crude 
2 yields 0.08 and 0.19 bbl of the same oils. Crudes 1 and 2 yield 0.13 and 0.18 bbl, 
respectively, of residuum, The reformer capacity is 10,000 bbl/day, and the ca- 
pacity of the catalytic cracker is 8000 bbl/day. On passing through the reformer, 
one barrel each of light, medium, and heavy naphthas yields 0.60, 0.52, 0.45 bbl, 
respectively, of reformed gasoline. On passing through the cracking unit, one 
barrel of light oil yields 0.28 bbl of cracked gasoline and 0.68 bbl of cracked oil, 
while one barrel of heavy oil yields 0.20 and 0.75 bbl of cracked gasoline and 
cracked oil, respectively. On passing through lube-oil processing, one barrel of 
residuum yields 0.45 bbl of lube oil. The capacity of the lube oil processor is 
1000 bbl/day. The refinery must supply at least 500 bbl/day of lube oil. The 
octane numbers of light, medium, and heavy naphthas are 90, 80, 70, respectively, 
while the octane number of reformed gasoline is 115, and that of cracked gasoline 
is 105. The octane number of aviation gasoline must be at least 107, while those 
of premium and regular motor fuels must be at least 94 and 84, respectively. 
The output of premium motor fuel is to be 0.45 times that of regular motor fuel. 
The vapor pressure of jet fuel must not be greater than 2 psi (pounds per square 
inch). The vapor pressures of light, heavy, cracked, and residuum oils are 
respectively 2.0, 1.2, 2.5, 0.1 psi. In the blending of fuel oil, Y2 = 0.3, V. = 0.4, 
Yr = 0.1 in Eq. (12-49). On a large sheet of paper write down the constraints 
for the refinery problem. Also write down the matrix of the coefficients. If the 
reader has access to a small digital computer, he will find it instructive to solve 
this problem by trying different sorts of objective functions such as those dis- 
cussed in the text. It is also interesting to study the changes resulting from an 
increase in the capacity of either the reformer or the cracking units. Finally, it 
is interesting to note what happens when one attempts to specify the output 
of several streams, such as aviation gas and fuel oil. 

12-32. An international petroleum company has m refineries and r crude- 
producing regions throughout the world. Each refinery and each producing 
region is close to a seaport. Tankers transport crude from the fields to the 
refineries. Different fields produce different types of crude. These are used at 
the various refineries in different proportions, depending on the type of equip- 
ment available at a given refinery. It is desired to schedule tankers for deliver- 
ing crude to the refineries. The time span to be considered is the coming year. 
Each refinery presents a schedule of requirements. The schedule specifies the 
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quantities of crudes wanted and the delivery date to be aimed for. To simplify 
the problem it will be assumed that all tankers are of the same size, and that 
refineries always order quantities of crude in integral multiples of a tanker's 
capacity. Thus the full load of any tanker will be discharged at only one refinery. 
After а tanker discharges its load, it can go to any port to pick up more crude, 
provided it can reach this port and from there a refinery in time to meet the re- 
finery’s schedule. We wish to determine the minimum number of tankers that 
will meet the scheduled demands at the refineries. Assume that the time re- 
quired to sail from any given loading point to a refinery is known and is not 
subject to any uncertainty. Set up the problem аз a linear programming problem 
and show that it is a transportation problem. Assume that tankers can begin at 
any loading port at any time desired and ean end their runs at any refinery. 
How does the problem change if the tankers must end up in specified ports? 
Hint: Arrange the demands at the refineries in order of increasing time, without 
taking into account which refinery is making the demand, Thus on day 29, 
refinery 1 may want a tanker load from field 3 and a tanker load from field 6, 
and the next time a demand occurs is on day 33 when refinery 4 may want a 
tanker load from field 4. In this way, one obtains a time sequence of N demands 
which must be met. Thus the ith demand specifies that a total of k tanker loads 
from perhaps several different fields must arrive at refinery s on а given day. 
Assume that once a tanker is loaded it immediately sails for the refinery where 
it is to unload, and the time required to make the trip is that referred to above. 
From this observation it is seen that the refinery demand schedule can be used 
to set up a loading schedule at the loading ports similar to the demand schedule 
at the refineries (simply by deducting the sailing time to the refineries plus 
loading time). ‘The jth loading item in this time-sequenced schedule will indicate 
that a total of w tankers must be available at port p for loading on a given day 
for delivery to perhaps several different refineries. Now consider the scheduling 
problem. A tanker which has just unloaded at a given refinery can be sent to 
any loading port if it can arrive there, load, and reach some other refinery in 
the required time. Then let zy; be the number of tankers available at the time 
and refinery associated with the ith demand which are sent to the port associ- 
ated with loading item ў to be loaded at the day required by item j. For the 
sake of conceptual simplicity, assume. that tankers immediately leave the ге» 
finery on unloading and arrive at the loading points on the day required. If 
the time required to sail to the loading port is less than. the time available, 
assume that the tanker remains at sea for this additional time. Note that the 
number of tankers used will be the sum of the slack variables at each of the des- 
tinations (loading items j), provided that tankers can be made available at any 
loading port and аб апу point in time desired. Hence, what; are the prices? 
Similarly, slack at the origins indicates that a tanker ends its run there. This 
type of problem is discussed in [5]. 

12-33. Solve the tanker; allocation problem formulated in Problem. 12-32, 
using the following data for three refineries and. producing fields, In the re- 
quirements matrix, the light-face numbers give the days on which a given re- 
finery wishes tankers to arrive from a given field, and the boldface numbers 
specify the number of tanker loads desired... Assume that loading and unloading 
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take a day. Tankers may start at any loading port at any time and end at any | 
refinery. Sketch graphically the optimal routing for each tanker. 


Requirements 


18 41 120 41 160 220 120 180 250 | 
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3 1 2 2 1 3 1 3 2 
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Sailing times (days) 


12-34. A painting contractor has been awarded the job of finishing a large; 
newly constructed office building. He is attempting to determine the minimum- 
cost paint blend which will meet all specifications. He has three grades of paint, 
Pi, Рә, Рз, and two types of thinner, T, T2, which can be blended together. 
The first requirement to be met concerns coverage. The final blend must be 

. such that only a single coat of paint will be needed. The contractor recognizes 
that the coverage depends on the viscosity of the paint. From previous experi- 
ence, he decides that any blend whose final viscosity is at least 350 centipoise (ер)! 
will do. The next requirement has to do with the “brilliance” of the finished job. 
This is a quantitative way of measuring reflectivity of the surface. The brilliance 
depends on the number of grams per gallon of a certain constituent X in the final - 
blend. For suitable brilliance, the final blend should contain between 10 and 
25 grams of X per gallon, Another characteristic of importance is the drying 
time. If the paint dries too slowly, a lot of dirt will become imbedded in the 
surface. If it dries too fast, it will later erack and peel off. The drying time de- 
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pends on the vapor pressure of the final blend. The contractor knows that for 
a proper drying time, the vapor pressure should be between 2 and 4 pounds per 
square inch (psi). The final specification concerns the "lasting quality" of the 
paint, i.e., it should remain in good condition for a certain length of time without 
fading. This quality can be measured by the number of grams per gallon of a 
certain constituent Y in the final blend. The final blend should contain at least 
550 grams of Y per gallon. A table listing the characteristics of the individual 
constituents which can be blended is presented below. Set up the problem of 
determining the optimal blend as a linear programming problem. Assume that 
all quantities of interest blend linearly by volume, and that the total number of 
gallons of paint required to do the job will be independent of the blend used. 
Solve the problem. 


Cost (dollars per gallon) 


Viscosity (cp) 

X-content (grams per gallon) 
Y-content (grams per gallon) 
Vapor pressure (psi) 


12-35. Solve the problem formulated in Section 1-3. How much can each 
of the profits be changed if the solution obtained is to remain optimal? 

12-36. Consider the problem of scheduling regular and overtime production 
of a certain item for the next year, on a month-by-month basis. The costs of 
producing one unit on regular time and overtime are $3 and $4, respectively, 
for months 1, 2, 3 and 7, 8, 9, and $2 and $2.50, respectively, for months 4, 5, 6, 
and 10, 11, 12. The cost of storing one unit for one month is $0.50. The initial 
inventory is 50 units, and it is desired to have 50 units in inventory at the end 
of the year. The regular and overtime capacities (in units per month) are 500 
and 200, respectively. The monthly demands are expected to be 400, 600. 500, 
800, 800, 700, 600, 800, 600, 700, 600, 800, in the order given. How should pro- 
duction be scheduled so as to minimize costs? 
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CHAPTER 13 


APPLICATIONS OF LINEAR PROGRAMMING TO 
ECONOMIC THEORY 


"I have found you an argument: but I am 
not obliged to find you an understanding." 


Dr. Samuel Johnson. 


13-1 Introduction. Almost all applications of linear programming are 
Somehow connected with economics. The difference between the material 
in this chapter and that of the preceding chapter lies in the emphasis. 
Chapter 12 emphasized practical industrial applications; now we shall 
concentrate on the applications of linear programming to economic 
theory. The dividing line between the material presented in the two 
chapters is rather fine. Some of the material which will be discussed 
here, such as the interpretation of dual problems, has potentially useful 
applications in industry. Similarly, the material on input-output analysis 
and its generalizations is considered by government agencies (where it 
is used) to be a practical application, although it does not find direct 
applications in industry. 


13-2 Theory of the firm; classical approach. The theory of the firm has 
long been a subject of study in economics, Interest has centered about 
the problems of determining the level at which a firm should operate 
(Le., how much should be produced) and of determining the optimal 
combination of resources to carry out production. The criterion used to 
solve these problems is the maximization of the rate of profit (say yearly 
profit). 

The case most frequently studied is that of pure competition. Pure 
competition implies that a large number of firms makes the same product, 
and that no firm can influence the market price. A firm selling in à 
perfectly competitive market сап sell any quantity at the given market 
price. For the moment, we shall limit our attention to a firm which 
produces a single good and markets it under pure competition. 

Let p be the market price for one unit of the firm’s product. If q units 
per year are produced, the firm will receive a revenue R(q) = pq. Note 
that p is independent of q. To determine the firm's yearly profit, on pro- 
ducing g units per year we must know the cost of production. The econo- 
mist, assumes that the yearly cost of production, C(g), is known for all g. 


416 


13-2] THEORY OF THE FIRM; CLASSICAL APPROACH 477 


He assumes that the cost function is based on the most efficient; tech- 
nological process of production, so that C(g) is a unique single-valued 
function of q. The rate of profit, II(q), associated with a production rate q 
is then 


Ig) = Rg) — C(4). (13-1) 


The firm wishes to maximize its profits for the year. Then if II(g) is 
differentiable, the calculus tells us that the production per year, g* > 0, 
which maximizes the profit rate is a solution to the equation 


di dR а dC ^ 
d^ d d ? ag п?) 


or q* = 0, or g* = c. It is almost always assumed that the q which 
maximizes II will satisfy (13-2). In the language of economies, (13-2) 
says that the firm continues production until the marginal cost dC/dg 
is equal to the price. 

'There may be more than one q which satisfies (13-2). In general, 
however, economists make the assumption that only: one of these will 
yield a non-negative profit. This g* is then the production rate which 
maximizes the profit rate. Note that q* will be a function of the market 
price p. A value of q* can be found for each p. The function g*(p) is the 
supply eurve for the firm. For any given market price, it shows how much 
the firm will produce in order to maximize its profits for the year. 

The above analysis has treated only part of the problem. Tt is also of 
interest to know how much of each factor of production the firm will 
use when it maximizes its profits. Assume that there are n factors, such 
as labor, raw materials, capital equipment, etc., which ean be used in 
the production of the good. Furthermore, we shall assume that pure 
competition also exists in the factor markets, so that the price ш; of one 
unit of any factor is a constant independent of the quantity purchased. 
If y; units per year of factor 7 are used, the cost of production is 


C= Y vac Кк, (13-8) 


where K is a fixed cost independent of ће level of production. 

In order to relate the output to the quantities of factors used in pro- 
duction, economists assume the existence of a production function 
q = $(Ui, ..., Yn) which, for any set of y; gives the maximum amount 
that can be produced by means of these quantities of the factors of pro- 
duction. Economists assume that this function is given, and that the 
production process will be such that it is always satisfied. Then if factor 
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2 is used at a level y;, the yearly profit of the firm is 


П = рф(у1,..., Yn) — У? wy: — К. (13-4) 
peri 
The profit rate can now be considered to be a function of n variables, 
the yi. 
Calculus tells us that the set of y; which maximize II should satisfy 
the set of n equations 


: am ag | 
ау: Рау; 


provided that each optimal y; lies in the interval 0 < y; < оо. These 
equations can be rewritten in the form 


ш; = 0, E S, n; (13-5) 


Wi wi 


= = ,س‎ LM EY a 13-6 
P = poy:  9qJoyi пагод quee 


In the language of economies, (13-6) states that each factor of production 
is used until the point is reached where its price divided by its marginal 
physical product is equal to the price of the product. Quite possibly 
one or more of the y; will be zero when II is maximized; that is, the abso- 
lute maximum of П may lie on the boundary of the region y; > 0, 
i= 1,..., n, over which the y; may vary. For such cases, we have 
equations like (13-5) only for those factors which are used at a positive 
level. To determine the absolute maximum of (13-4), we must find, 
in addition to the solution of (13-5), the relative maxima of (13-4) for all 
combinations of one or more y; — 0, and select the solution yielding the 
largest value of II. 
Economists often assume that (13-6) defines a set of functions, 


Yi = gius... wn р), i—lh...ym, (13-7) 


which are the demand functions for the factors of production. The demand 
for any factor of production is, in general, dependent on the prices of all 
the factors of production and the price of the product. Intuitively, the 
qm of (13-7) means that Eqs. (13-6) can be solved explicitly for 
the yi. 

It should be noted that, for the above analysis to be meaningful, the 
production function must be continuous and capable of being differentiated 
everywhere. This is a strong assumption. Of course, from the practical 
point of view, the determination of a production function is often ex- 
tremely difficult. 

In the above derivation, we have been a little vague in indicating 
whether there were certain factors of production, such as the size of 
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the plant, that were not allowed to vary. The above analysis assumes 
that everything can vary, and hence it also determines the size of the 
firm. However, where the plant size is fixed, the same sort of analysis 
can also be applied in the short run. Formally, the same equations are 
obtained. Only the nature of the production funetion and the number 
of variable factors change. For the short run, there is an upper limit on 
the productive capacity, and thus the optimal produetion rate may not 
satisfy (13-2); instead it may be an end point, i.e., the maximum possible 
production. 

In the theory of pure competition it, is usually assumed that any one 
firm never makes more than a single product unless there are joint, prod- 
ucts, in which case one item cannot be made without producing one 
or more other goods. If a single firm did produce more than one good, 
the production of each good would be treated as if it were made by a 
different firm, although the same preduction facilities were used to produce 
all products. This can be done because of the assumption that each 
factor of production can be varied continuously so that. there is no need 
to be concerned about the fact that machines, ete., come only in certain 
discrete, or “quantized,” sizes. It is imagined that a machine of precisely 
the required capacity is always available and that its price is proportional 
to its capacity (or, more generally, to some function of its capacity). 

Let us examine the case of joint products, where a single firm must 
produce all joint products. However, by varying the mix of factors of 
production, it may be able to alter the mix of the joint products. Assume 
that there are m joint products. Let g; be the yearly production of prod- 
uct j and y; be the rate at which factor i with price w; is used in the 
production. 

For each product it will be imagined that there:exists a production 


function 
4j = ej Vn); (13-8) 


so that once the mix of the factors of production is specified, the rate of 
production of each good is uniquely determined. Then for any set of Yi, 
the firm's yearly profit is 


п = >> pies --- Yn) — Dy wai К. (13-9) 
j=l per 
The set of y; which maximizes the profit should then satisfy the set of 
equations 
а Oo; ЖАЙ LA 
2, Prag TT O £5, iT (13-10) 


j=l 


provided that the optimal value of each y; lies in the range 0 < y; < co. 


480 LINEAR PROGRAMMING AND ECONOMIC THEORY [снАР. 13 


If one or more y; should be zero, (13-10) applies only to those which are 
positive. These equations determine the rate at which each factor is to 
be used. The production functions enable one to determine the rate at 
which each product should be produced in order to maximize the firm's 
profit rate. 

Having given a brief survey of the theory of the firm under pure com- 
petition, we shall next consider the monopolistie case. Here it is assumed 
that the firm either controls a sufficiently large share of the market or 
has a product which is sufficiently differentiated from those of other 
firms that it can set the price. Furthermore, the sales volume will depend 
only on that price. For any given price p, consumers will purchase at 
arate q(p). The function q(p) is called the demand curve for the product. 
Here the firm has to face the problem of determining the price p such that 
if the production rate is equal to the rate of demand at that price, the 
profit rate will be maximized. The yearly profit will then be a function 
of the price p. 

As before, we assume the existence of a cost function C(g). If the 
price is p, q(p) units will be produced because precisely (р) units can be 
sold. The yearly profit is 


П(р) = pq(p) — Cla(p)I. (13-11) 

The price p* which maximizes the profit rate should satisfy the equation 
dis dg dO dq _ 8 

qa + ат ape (13-12) 


The production rate which maximizes the profit rate is then q(p*). To 
obtain (13-12) we must assume that the demand and cost functions are 
differentiable. 

It should be noted that, unlike iu the case of pure competition, there 
is no supply curve in the monopolistic case. The firm is free to set, the 
selling price, and there is only one price that maximizes yearly profits. 
Thus the supply curve is a single point. The firm will not consider any 
price other than that which maximizes the profit rate. 

We have yet to consider the oligopolistic case, where a relatively small 
number of firms produces goods which are close substitutes. Each firm 
is free to choose the selling price of its product. Now, however, the demand 
for a given firm’s product is a function not only of the price of that par- 
ticular product, but also of the prices of all other substitute products. 

Here we have encountered a case where the criterion of profit maxi- 
mization does not lead to a uniquely determined solution. No firm can 
maximize its profit independently of what the other firms do. The final 
result depends on the strategies followed by the various firms. : A solution 
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can be obtained if it is assumed, for example, that the firms act in such a 
way as to maximize the total profit of all firms. A different result may be 
obtained for each different strategy. $ 

To terminate this brief survey of the classical approach to the theory 
of the firm, several points are worth noting. First, since the approach 
is essentially static, and since no interaction exists between producers 
and consumers, the maximization of the profit rate at each instant of 
time is equivalent to maximizing the total profit over all time. Secondly, 
in the case of pure competition, the production of a finite, determinate 
quantity depends on the assumption that the marginal cost of production 
increases when q is larger than some given value, i.e., costs are not directly 
proportional to the quantity produced, for example. 


13-3 Linear programming and the firm. Linear programming provides 
a much more practical approach to the theory of the firm than does 
classical analysis, which is based on the assumption that a firm produces 
just a single good. The linear programming model makes it possible to 
be more realistic and to permit the firm to produce a number of products 
which jointly use the production facilities. This approach is more real- 
istic because it allows recognition of the fact that various types of pro- 
ductive resources are available only in certain discrete sizes, so that it 
may be necessary for a firm to produce a number of products in order 
to make best use of the productive facilities and thereby maximize profits. 

Linear programming typically treats the firm from a strictly short-run 
point of view. Production facilities are assumed to be fixed. These limited 
production facilities limit the total quantities of the various items which 
can be produced. In addition, of course, there may be limits on the 
quantities available of the various raw materials. 

Instead of dealing with the notion of a production function which is 
everywhere continuous and differentiable, linear programming starts at 
a more basic level with various processes of production, Suppose that 
there are m factors of production (resources) whose supply is limited. 
For a given process of production, let a;; units of resource 1 be required 
to produce one unit of good j. The vector a; — laij, - . +) am;] tells how 
much of each resource is required to make one unit of good j. Previously, 
we have referred to a process of production as an activity and to the 
vector a; as an activity vector. This is thé accepted terminology in 
economies. 

The linear programming model assumes that the number of activities 
for making a given good is finite. This is different from the classical 
approach. The concept of a production function allowed for an infinite 
number of ways to produce a product. If we produce z; units of good j 
(j refers to a given good produced by a given production process), x; is 
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called the level at which activity j is operated. In the linear programming 1 
model, the firm chooses the level at which each activity is to be operated b. 
in such a way that profits will be maximized. The various activities imply 8 
a certain type of production function for апу one good. Let us examine _ 
the nature of this production function in a little more detail. n 
Tora given output 7, the classical production function] = $(yi,. .., Yn) _ 
implied that any point in the non-negative orthant on this surface in Ё" 
gives a set of inputs which, when properly used, will yield g units of the 9 
good. Linear programming, however, is not so general Let the subs А 
script k refer to those activities which can be used to produce a given j 
good. Then if activity k is operated at a level zy, any set of x, > 0 such 


ja @= XU (13-13) | 


will yield the required amount of the product, If y = [y, . .. m is 
a vector giving the quantities of the resources needed to produce g, then ~ 


у = Dna. (13-14) 


The y computed from (13-14) for any set of zp satisfying (13-13) will do. 
If we write y* = [y, 1), af = (ax, 1], Eqs. (13-13) and (13-14) can 
be combined to read 


* * А " 
у*= ла alz 0. (13-15) 


The set of points y* satisfying (13-15) forms the convex polyhedral cone 
generated by the az. Let b; be the maximum amount of resource 7 avail- 
able in the time period under consideration. Then if b = [bı,..., bn], - 
any y* given by (13-15) for which y < b is an allowable point on the. 
production function. Thus the production function is the collection of 
all points y* in the convex polyhedral cone generated by the aj for which. 
y € b. This production function does not have the continuity and 
differentiability properties postulated for the classical models. 

For a firm selling under pure competition, the linear programming 
problem whose solution yields the levels at which each activity should be 
operated to maximize profits will have the form 


Ax Sb, x20, maxz=cx, : (13-10) 


All inequalities will be of the form < because the constraints represent 
limitations on the available resources, Since the firm can sell its entire 
output at the prevailing market price, no constraints on the quantities 
produced appear. Each c; represents the profit on one unit of the good 
made by activity j. 

We know that an optimal solution to (13-16) need not have more 
than m of the z; > 0. This means that the firm can maximize its profits, 
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using only m of the activities. The remaining activities will not be used 
at all. It may of course be true that an optimal solution will require 
that a given produet be produced by two or more activities. For any 
optimal solution, z; — c; > 0 for all j. Now c; — 2; can be interpreted 
as the increase in profit made possible by producing one additional unit 
of j and adjusting the levels of the other basic activities so as to obtain 
a feasible solution. Thus, provided that degeneracy is eliminated, a basic 
feasible solution is optimal if it is not possible to inerease the profit by 
inserting into the basic solution any activity not in the solution. These 
fundamental theorems of linear programming are the equivalents of 
Eqs. (13-2) and (13-6) for the classical approach. s 

Our discussion of refinery operations in Chapter 12 showed that linear 
programming can be used in industries where there are joint, produets 
and where the product mix can be varied by changing the nature of the 
process, Constraints on the quantities to be produced can also be handled 
without difficulty. However, linear programming cannot handle the 
details of the monopolistic’ case, where the quantity which can be sold 
depends upon the price. Such a situation leads to a nonlinear programming 
problem. Of course, linear programming cannot help solve the oligopolistic 
case either. In practice, linear programming has been applied to both 
monopolistic and oligopolistic industries. However, the prices are then 
usually assumed to be constants, and the demand is made independent 
of prices. 

It is important to note that if pure competition is treated by linear 
programming, it is necessary to assume that the profit from any one 
product is directly proportional to the quantity produced. If marginal 
costs change with the level of production, the objective function will not 
be linear, and we must solve a nonlinear programming problem. Thus, if 
we drop the assumption of pure competition or if the marginal costs 
change with the level of production,* we leave the realm of linear 
programming. 


13-4 Economic interpretation of dual linear programming problems. 
In the preceding section, it was shown that for a firm with constant 
marginal costs, selling in purely competitive markets, and having only 
a finite number of activities, the problem of determining the level at 
which each activity should be operated in order to maximize profits is a 
linear programming problem of the form 


Ax <b, x20, maxz- cx. (13-17) 


There is a constraint for each resource in limited supply. 


* See, however, Problem 12-10. 
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The physical dimensions of the variables z; will be the units of some 
good produced for some given time period. The dimensions of the b; 
are units of resource 7 available in a given time period, viz., machine 
hours per week. The a;; then have the dimensions of units of resource 7 
per unit of good j. 

Consider now the dual of (13-17). It is 


A'w 2 с’, w>0, min Z = b'w. (13-18) 


Since the dimensions of the c; are dollars per unit of good j, a;;w; must 
have the dimensions of dollars per unit of good j: But the dimensions 
of aij are units of resource 7 per unit of good j. Thus the dimensions of 
w; must be dollars per unit of resource 7. 

To each resource ? there corresponds a dual variable w; which, by its 
dimensions, is a price, or cost, or value to be associated with one unit 
of resource 7. Suppose that we assign a value w; to resource 7. Then 
Z = b'w is the total value of-the available resources. The jth dual con- 
straint reads ZZ аро; > cj. But XFL, aijw; is the value of the re- 
sources used in making one unit of product j. The dual problem determines 
the w; so that the total value of the resources is minimized, and the value 
of the resources used in producing one unit of j is at least as great as the 
profit received from selling one unit of j. 

The dual variables w; are sometimes referred to as imputed values or 
shadow prices for the resources. Note that the dual variables have nothing 
to do with the actual costs of the resources. The c; are profits, and thus 
the actual costs of the resources never appear. Instead the dual variables 
provide, in a certain sense, a way of measuring the contribution to the 
profit c; of each resource û. Let us see why this is so. Recall that if z; > 0 
is in an optimal solution to (13-17), then by complementary slackness 
ZF, aijw; = с; so that for the activities used, the value of the resources 
used to produce one unit of j is precisely equal to the profit. We know also 
that if in an optimal solution to (13-17), the ith constraint is a strict in- 
equality (the corresponding slack variable is positive) so that not all of 
resource 7 is used, then w; = 0, and the cost or value of that resource 
is 0. Such a resource is a free good. 

The valuation of the resources by means of the w; is thus an opportunity- 
cost valuation. To see this more clearly, recall that for optimal solutions 
to the primal and dual problems, z = cx = Z = b'w. The maximum 
profit is equal to the minimum value of the resources. Now if it were 
possible to increase or decrease the amount available of resource i by one 
unit without changing the solution to the dual, the maximum profit would 
be increased or decreased by w;. This is the basis for the opportunity-cost 
interpretation. Of course, if we actually change b; to b; + 1, the profit 
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will not, in general, increase by ш; because the entire optimal solution 
changes. However, when z is being maximized, w; is a measure of the 
rate of change of z with respect to b;. When a resource is not fully utilized 
in an optimal solution, z will not change if the availability of the resource 
is changed slightly. Hence w; for this resource should be zero. The com- 
plementary slackness principle ensures that this will, indeed, be the case. 

The dual variables have potential applications in cost accounting. 
Consider a large decentralized corporation which is broken down into a 
number of departments. Each department may make several products. 
In addition, there may exist a number of different activities for making a 
single product. These various departments jointly use manufacturing 
facilities and other services or resources which are in limited supply. 
Suppose that the chief executive officer has obtained an optimal solution 
to the linear programming problem for the entire corporation, and there- 
fore knows which activities should be used and what their level should be. 

Top management wishes to make sure that the department managers 
select the proper activities. However, this selection process should originate 
with the department managers and not come about as а result of direetives 
from top management. Suppose that to each resource which is in short 
supply we assign the cost wi, where w; is the ith dual variable obtained 
from the optimal solution to the corporate problem referred to above. 
For each unit of resource 7, a department manager must pay w; (regard- 
less of the actual cost of 7). Then the cost of one unit of good û produced 
by activity j is 20:0: = 2j. We now suppose that the department 
manager is paid c; for each unit, cj being the unit profit on j. If 25 > су 
the department manager will find that his department is losing money 
when activity j is operated at a positive level. Consequently, the manager 
will be forced to avoid using activity j. He can find no activities for 
which z; < cj, and hence will automatically seek out activities for which 
г; = cj, i.e., activities that should be operated at a positive level. There 
remains the problem of getting the managers to operate these activities 
at the correct level. This cannot be done by means of the above costing 
procedure, and hence another approach must be used. For a unique, 
nondegenerate optimum, the activities for which г; = с; form a basis. 
Thus only one thing can happen, i.e., the resources are not fully utilized. 
If the resources are utilized to the fullest extent. possible, the activities 
must operate at the correct level. Thus it is only necessary to ensure that 
the available resources are used, and the production pattern will be 
optimal. When there are alternative optima, it is still true that the solu- 
tion will be optimal if the resources are utilized to the fullest extent possi- 
ble. In this case, however, the manager's optimal solution may not be 
basic. 

‘An economic interpretation of the dual variables is possible for other 
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types of problems. Consider the diet problem discussed in the last section 
of Chapter 12. In its simplest form, this problem can be written 


Ax > b, x 2 0, 
min 2 = cx. 


Each constraint says that at least a certain amount of some nutrient 
must be consumed. The dual problem is 


Ау < с’, w > 0, 
maxZ = b'w. 


The dual variable w; has the dimensions of dollars per unit of nutrient 7 
and can be interpreted as the value of one unit of nutrient 7. 

The jth constraint of the dual problem reads: 377 , ази; < cj. But 
ZL, aijw; is the value of the nutrients in one unit of food j. The dual 
problem thus determines the w; in such a way that the imputed value 
of the minimal diet is maximized, subject to the constraints that the 
value of the nutrients in one unit of each food is less than or equal to 
the cost of the food, As expected, an optimal diet uses only foods for 
which the value of the nutrients is equal to the cost. 

The dual variables in the transportation problem can easily be inter- 
preted as prices also. To do this, it is convenient to formulate the primal 
problem so that the dual variables are non-negative. We write the primal 
as 


n 
= Убаи ар t=1,...,m, 
jul 


Dd zu > b; iSd oam (13-19) 


i=l 
minz = cx, 
If Xa; = by, it is clear that the strict equalities must hold for any 


feasible solution. 
The dual of (13-19) is 


"j — S оа, — alij, 
(13-20) 
maxZ = by — au, 


and the dual variables are non-negative. If we write the dual constraints 
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as vj < с;з + u;, each и; can be interpreted as the f.o.b. value of the 
product at origin 7, and v; as its value after delivery at destination j. 
The dual prices are determined in such a way that the value at destination 
j is never greater than the value at origin i plus transportation cost from 
i to j. In an optimal solution, only those routes are used for which 
vj = с;з + Wi. 


13-5 Input-output analysis. We now turn from the study of an individ- 
ual firm to the study of an economy, which involves the interactions of 
2 number of industries. 

A very interesting and important model of interacting industries 
(frequently called input-output analysis) was developed by Professor 
Wassily Leontief in the early 1930’s. Since its introduction, a great deal 
of theoretical development and some testing have taken place, so that 
it ranks today as one of the most, important models of an economy. 

Leontief imagines that the economy consists of a number of interacting 
industries. Each industry produces a single good and uses only one 
process of production to make this good. In fact, an industry can be 
thought of as a process of production. To produce its good a given in- 
dustry needs as inputs goods made by other industries, labor, and perhaps 
other inputs from outside the system. Each industry must produce 
enough to supply the needs of other industries and to meet the external 
(exogenous or net) demand, The external demand includes consumer 
demand and may also include government demand and foreign trade. 

We shall suppose that there are n industries in our hypothetical economy. 
For a given time period (say one year) let yı; be the amount of good i 
(from industry i) needed by industry j. Also let b; be the exogenous 
demand for good 7. The total amount х; which industry i must produce 
to meet exactly the demands is 


LJ 
a= yai t= 1.6m (18-21) 
jel 


Next we must relate the inputs of y; to the output zy for each in- 
dustry j. This means that we must develop the production functions. 
Leontief assumed a very simple type of production function. He supposed 
that the required input of any factor was directly proportional to the 
production, ie. . 

yg = ал» alij, (13-22) 


where the a;; are constants. Thus the factors of production are used in 
fixed proportions. Furthermore, there are no decreasing returns to scale. 
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When (13-22) is substituted into (13-21), we obtain 


n 
= agb i—1...,m (13-23) 
ja 
or 
У) (ûû — аз =, i-1...,n 
j=l 
or 
٤ (I — А)х = b, (13-24) 
where A = |а|, x = (zı, ..., anl, b = [b1,..., bn]. If the exogenous 


demands represented by b are given, then (13-24) is a set of n simul- 
taneous linear equations to be solved for the amounts z; which must be 
produced to meet these demands. For a meaningful system we expect that 
there is a unique solution. Later we shall see that, under all normal 
circumstances, r(I — A) = m. It is also necessary that each т, be non- 
negative. It will be shown that this also follows automatically for all 
meaningful economies. 

The aj; are the technological coefficients which describe the nature of 
the economy. The dimensions of the aij depend on the dimensions in 
which the уу, 2; are measured. The natural dimensions of xj are units 
of product j, and similarly, the natural dimensions of yi; are units of 
product 7. In such a case, the dimensions of а; are units of product f 
per unit of product j. Frequently, the Zi, угу are given in monetary units 
во that ху, yi; have dollar values. Then the dimensions of a;; are dollars 
of i per dollar of j. 

If we have an economy of the type postulated by the Leontief input- 
output system, (ће а; can be computed as follows: We choose a given 
time interval, say, one year. A table is constructed which lists in row f 
the amount of product i sold to industry j, = 1,...,n, and the exoge- 
nous consumer demand. In general, the entries will be in dollar terms 
although the natural dimensions may also be used. By summing row 7, 
we find the total output z; of industry i for the period under considera- 
tion. Such a tableau is shown in schematic form in Table 13-1, Each dij 
is found by dividing y; by z; (not z;).* The Leontief system is static 
and does not provide for technological improvement or other factors 
which would cause the aj; to change with time. 


* Here we assume that everything used by industry j is produced in the period 
under consideration, and that all of the production of industry i is used. In 
attempting to apply input-output analysis to the real world, we must be careful 
to account for changes in stocks, i.e. inventories. 
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TABLE 13-1 


Industry 


1 ша yi2ttt Yin 


2 ym ym" yas 


If the inverse of I — A exists, then (13-24) may be solved, and the 
unique solution is given by 1 


x = (I — A)7!b. (13-25) 
Denoting the elements of (I — А)! by оу, we can write 


"TT $1.27) (13-20) 


jmt 


there exists an x > 0 satisfying (13-24). The only type of bill of goods 
which is of economic interest must satisfy b > O.and b. » 0, і.е, at least 
one by > 0. The reason for this is that b includes consumer demand, 
and hence all b; cannot be zero. Later we shall consider a modified model 
in which b can be 0. Thus, in order that there exist at least one producible 


the е, — a, and the non-negative orthant of E" contains at least one 
point w € C such that w > 0, w »* 0. Then w will be a producible bill 
of goods, as will any point b = Aw, А > 0. The set of all producible 


economy described by a Leontief model, ie., the only productive factor 
which is not produced by the economy. For the economy described by 
(13-24), assume that-aojz; units of labor are needed to produce ту units 
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of good j; ao; is a constant. Given a set of x;, the total labor requirement 
is then 577, doj; = aoX, ao = (а01,..., don). It is easy to compute 
the labor necessary to produce any final bill of goods b since the gross 
production required to meet this bill of goods is x = (I — A)-!b. The 
labor needed to produce b is [ao(I — A)-!]b. 

Suppose that there is an upper limit z to the labor available in the 
economy. For b to be produced, it is necessary that 


[8001 — A)^!]b < z. (13-27) 


Thus when labor is a limited resource, the set of producible bills of goods 
for an economy with a technology matrix A is the intersection of three 
Sets: C, the half-space (13-27), and the non-negative orthant of E". 


13-6 Prices in a Leontief system. Nothing has been said so far about 
prices in our hypothetical economy. Let p; be the unit price of good j. 
Then if the a;; are measured in their natural dimensions, a;;p; is the cost 
of the a;; units of good i required to make one unit of good j. The cost 
of goods 1, ..., n needed to make one unit of good j is Ур, mpi We 
shall call the difference between the price of good j and this cost the 
“value added” by industry j in producing good j. Denote the value added 
by r; Then, by definition, 


n 
Pi Уат т G=1,.-.,0, (13-28) 
i=l 
or in matrix form 
(I — A)p =r, (13-29) 


where p = [р,..., Pn], T = [r1,..., ra]. Now if I — А is nonsingular, 
so is (I — A)’. Hence if we know the value added by each industry, 
it is possible to obtain unique values for the prices in the economy. Re- 
calling that the inverse of the transpose is the transpose of the inverse, 
and using (13-26), we can write 


n 
D;— ys «Uu J = ES (13-30) 
i=l 

The value added can be broken down into labor costs and other factors 
(such as profit). Imagine that only a single wage rate, w, exists in our econ- 
omy. Then if ao; units of labor are required to produce one unit of prod- 
uct j, the labor cost for one unit of product j is аоуш, and hence т; can be 

written 
Tj = o + 8j, (13-31) 


where з; refers to the value added other than labor. 
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Let us consider àn economy in which labor is the only primary factor. 
Suppose that we wish to determine a set of accounting prices which 
reflect only labor costs in the value added (no profits, etc.). These may 
be thought of as the prices which would exist if the economy operated 
under pure competition and in a state of long-run equilibrium. Then 
т = шау, and from (13-29), 


p = wag(I — A). (13-32) 


If the ууу, ху are measured in monetary terms, they include the prices. 
Denoting by aj, aj; the technological coefficients in their natural and 
monetary dimensions, respectively, we see that 


а= (Шуа, (13-88) 


Since we expect XFL, pia]; < р; because there must be labor costs in 
the production of good j, it follows that af; > 0, and rn a; < 1. 

The duality of prices and quantities in a Leontief system should be 
apparent to the reader. It is instructive to examine this aspect in more 
detail. Consider the linear programming problem 


(I—A)x>b, x29, тіп 2 = aX. (13-34) 


This problem seeks to minimize labor requirements while producing at 
least the bill of goods b. The unique optimal solution is given by (13-25), 
with the minimum labor requirement being а0(1 — A)~'b. Thus the 
Leontief model can be considered to be a very simple linear programming 
problem. Now the dual of (13-34) is 


(= Аун < а w20  maZ-bWw. (19-35) 


The unique optimal solution to the problem is w = (I — IND Ao: 
From (13-32), w = p/w, so that, once the wage rate is known, the dual 
of (13-35) determines the prices in the economy provided the yalue added 
is entirely due to labor. Hence, the problem of determining the prices in 
a Leontief economy can also be considered to be a linear programming 
problem, It is the dual of the problem which determines the quantities 
to be produced. 

Duality theory shows that if the bill of goods b is producible, then 
there exists a set of non-negative prices for the economy. Since these 
prices are independent of b, it also follows from duality theory that the 
same prices will apply for all producible bills of goods. 

The duality of quantities and prices can also be interpreted in terms 
of convex cones. The prices in the economy are given by (13-32). Hence, 


pb = wal — A) (I — A)x = waox > 0, (13-36) 
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provided b is producible and w > 0. If b is producible, it lies in the 
convex polyhedral cone C spanned by the columns of I — A. But by 
(13-36) p is an element of C*, the cone polar to С. Now a theorem on 
convex cones (Chapter 2) states: If C contains no vector y « 0, С* con- 
tains a y > 0, y #0. We do not expect the cone C spanned-by, the 
columns of I — A to contain a b. < 0, for this would mean that negative 
net amounts could be produced, although the gross amounts are non- 
negative. Thus we expect C* to contain a p > 0, and hence we see from 
the properties of cones and from the duality theory of linear programming 
that there should be a set of non-negative prices for the economy. 


13-7 Other properties of the Leontief system. In Chapter 2 we noted 
that if limk» AF = 0, then 


ductos SEA Wl АНАР... (13-37) 


ke 


In the preceding section, we saw that if monetary units are used, then 
it is always to be expected that Y^; аў; < 1 for all j. Furthermore, it 
was shown in Chapter 2 that if this condition holds for non-negative аў» 
then the expansion (13-37) is valid, and we can write x = (I — A)~'bas 


x=b+Ab+A°%b+---. (13-38) 


When the а;; are expressed in their natural dimensions, there is no 
reason why >; aj; < 1 should be expected to hold. However, we suspect 
that the existence of the power-series form of the inverse should be inde- 
pendent of the dimensions used, and hence (13-38) should hold also 
when natural dimensions are used. There is an interesting physical inter- 
pretation of the power-series expansion which shows that the expansion 
should always be valid. Equation (13-38) is nothing more than à mathe- 
matical representation of the fact that the gross production can be thought 
of in the following way: First, the gross production must consist of the 
final bill of goods b. To turn out b, each industry 7 must produce an 
additional quantity 5^; а,б; to be used in the production of b by industry 7 
and other industries, i.e., Ab must be made in addition to b. However, 
to produce the amount Ab, an additional quantity A(Ab) — A?b must 
be produced. To produce А?Ь requires an additional output A?b, ete. 
"Thus the total required production is (13-38). 

Under all normal circumstances, A*b — 0 for any b ог A* > 0 as 
Е — co, that is, we always expect the series (13-38) to converge. This 
will be true if less than one unit of good 7 is required in total by all in- 
dustries to produce one unit of good j. For example, suppose that there 
were only two industries, steel and coal. Tf the first industry is taken to 
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be coal, then consider the first, component of (13-38). The first component, 
of b is the consumer demand for coal. The first component of d = Ab is 
the coal needed to mine 5, units of coal.and to make ba units of steel. The 
first component of Ab is the coal needed to mine d; units of coal (needed 
to mine b; units of coal and to make ba units of steel) and to make d» 
units of steel (needed to mine b, units of coal and to make b» units of 
steel), etc. The series will converge if only a finite amount of coal is 
needed to produce b, units of coal and bs units of steel for consumer 
demand; the same is true for the required quantities of steel. These 
conditions should always hold, and therefore, we should always be able 
to write (13-37). 

The series expansion (13-37) reveals some other points of interest. 
Since ау > 0 for all 1, j, irrespective of the dimensions used, we see 
that every element of (I — A)T} will be non-negative for a Leontief 
system. This in turn means that for any b > 0,x > 0. Hence the ' 
convex polyhedral cone C spanned by the columns of I == A must contain 
the entire non-negative orthant of E". In other words, any bill.of goods 
is producible if the labor supply is unlimited, If z is the upper limit to 
the labor supply, then any bill of goods satisfying ao(I — A):!b < г 
is producible. 

Our above discussion may leave the impression that a Leontief economy 
requires all factors of production to be used in fixed proportions, and does 
not permit possible substitutions. However, this is not the case. Pro- 
fessor Samuelson was the first to make the interesting observation that 
if there is only one primary factor, labor, then the Leontief economy is 
compatible with the notion of substitutability. Although substitution 
is physically possible, it will never be observed because it is ruled out by 
the following economic considerations: If the only value added is the cost 
of labor, then only the wage rate can be freely varied. The prices of the 
goods are directly proportional to the wage rate, but the price ratios 
pi/p; are invariant and do not depend on the wage rate. Thus if the wage 
rate goes up, à shift from labor to machines will not occur because the 
increase in the wage rate also increases the price of machines. 

We shall not prove rigorously that substitution, although physically 
possible, is ruled out. Several proofs may be found in Koopman’s work [3]. 


13-8 The aggregation problem. In any attempt to apply a Leontief 
system to a real economy, we are immediately faced with a number of 
problems, in particular, with the question of aggregation. l'or an economic 
system such as the U.S. economy, there are thousands of different ac- 
tivities which, in a Leontief model, could be considered to be industries. 
The task of collecting the data needed to estimate the technological 
coefficients for an economy of several thousand industries would be 
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monumental. Furthermore, even if the data could be collected, none of 
the computers available today could solve the resulting system of equa- 
tions. Thus, to apply the Leontief model to a real economy, it is always 
necessary to aggregate a number of activities into a single industry. 

The problem of aggregation is very difficult, and no definitive treat- 
ment of the subject has yet been given. Here we shall do no more than 
indicate the nature of the problem. It should be immediately obvious 
that the method of aggregation can have an important bearing on the 
answers obtained from a Leontief model. Three of the most frequent 
criteria used for aggregation are: (1) substitutability, (2) complementarity, 
and (3) similarity of production functions. If the first criterion were 
adopted, we would aggregate products which are close substitutes. Ac- 
cording to the second principle, we would aggregate products that comple- 
ment one another and are used in roughly fixed proportions, According 
to the third criterion, we would aggregate items requiring essentially the 
same type of production process. None of the above techniques provide 
a foolproof method of aggregation because it is almost impossible to find 
perfect substitutes, exact complements, or identical production functions. 

Let us study briefly what happens to the system of equations (13-24) 
if we aggregate a set of industries. To simplify the discussion, we assume 
that all quantities are measured in monetary terms. Suppose that we 
decide to aggregate the industries m, т + 1,...,n into a single industry. 
If Û, is the net demand for this single industry, then 


bn = x bi, (13-39) 


iam 
The sales from aggregated industry m to any other industry j will be 
denoted by mj, and 
n 
Ўн = D> yi. (13-40) 
dm 
Similarly, if Jim represents the sales from industry 7 to the aggregated 
industry m, then 
n 
Dim = YD yi (13-41) 
j-m 


If, for the period under consideration, the gross sales of industry j are 
gates by z;, апа £, denotes the gross sales of the aggregated industry, 
then 


2 
+ = D> z; (13-42) 
j=m 
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The new technological coefficients associated with the aggregated in- 
dustry are 


j 1 е ^ 
вы = LD yy = aly jeloom-i (18-43) 
йт =m 


and 


j ү; ‚2 2 
fave gg ee = шош 1o 4=0,...,m. (13-44) 
я Зет tj im jm Tj 


Unfortunately, the @ are not independent of the bill of goods... Hence, 
if these coefficients are determined under one set of conditions, it does 
not follow that they will remain valid for a different bill of goods. 

If the aggregated industries have precisely the same production func- 
tions so that aj, = aj, j, k = m,...,n, then the df, are indeed con- 
stants, and d$, = аб. Furthermore, if the aggregated industries are 
exactly complementary so that ту = fjr, j = m,...,n—1, then the 
4;,, are again constants since 


However, if the industries are exact substitutes for one another, but the 
production functions are not identical, it does not follow that the dj, 
will be constant unless some specification is made as to the proportions 
in which the various substitutes will be used, 


13-9 Decomposable systems. Several interesting features of a Leontief 
economy often become immediately obvious if we draw a graph repre- 
senting the flows in the economy. ‘The nodes refer to the industries. 
И aj > 0, we draw an oriented branch from i to j. The number of 
branches joining i and j may be 2, 1, or 0, In the event that ау > 0, 
and aj; > 0, there will be two oriented branches (in opposite directions) 
joining i and j. Industry j buys directly from industry 7 if the network 
contains the oriented branch (i, j). We say that j buys indirectly from i 
if there is an oriented path from i to j containing two or more branches 
in the path. An industry j buys indirectly from i if it buys from one or 
more industries that are supplied directly or indirectly from i. Industry j 
may buy both directly and indirectly from i. An industry j does not buy 
either directly or indirectly from i if there is no oriented path leading 
from i to j. 

An economy is said to be indecomposable if each industry buys directly 
or indirectly from all other industries; otherwise the economy is called 
decomposable. In an indecomposable economy, a change in the net require- 
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ment for product j will be felt throughout the entire economy; i.e., if 
there is an increase in the net demand for product j, while the demand 
for all other products remains unchanged, then a strict increase must 
occur in the gross production of every product since industry 7, which must 
increase its gross production, directly or indirectly uses products from all 
other industries. Hence, every industry must increase its gross production. 

For any indecomposable economy, we can make an additional observa- 
tion concerning the matrix |o;j| = (I — A)™™. Recall that x is given 
by (13-25). If b is replaced by b + Xe; and the new gross production is 
x-+ Ax, then x + Ax = |le;jl(b + ej) or Ax = Mle;jlej. , Hence 
Ax = Лај a; = [a1;,..., anj]. We noted in the above paragraph that 
if A > 0, Ax > 0, and therefore œ; > 0,7 = 1,...,n. This shows that 
for an indecomposable economy, each element of (I — А)! is strictly 
positive. 

Let us next study briefly decomposable economies. Suppose that an 
economy is represented by a graph, such as that shown in Fig. 13-1. 
The graph for this economy decomposes into three disjoint graphs which 
are not connected. If we number the industries as shown, the technology 
matrix A for this economy can be written: 


А, 0 0 
ај 012 
A—|0 А 0); a= [os 2S 
‘LO 0 Аз 


drab а55 56 457 
А, AS d ‘| ; Аз = |авѕ 6 de7|' 
* бт ату 76 477 


Industries 1 and 2 neither buy from nor sell to the remaining industries. 
The same holds true for industries 3, 4 and 5, 6, 7. Such an economy can be 
viewed as three separate economies, that is, (I — A)x — b breaks down 
into (I — A;)x, = bı, (I — Аз)х = bs, (I — Аз)хз = bs. An economy 
is ealled completely decomposable if the technology matrix can be written 


A, 0۰۰.0 
дее dean gies thy (13-45) 
EN aed SA 


where m > 2. Such an economy can be treated as m separate economies. 
When the graph of an economy is drawn, it is immediately obvious whether 
or not it is completely decomposable. 

We wish to note that if the industries had been numbered differently : 
in Fig. 13-1, the technology matrix would not have had the form (13-45). 
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Fieure 13-1 
1 


Pec re ен 
ү ° 


5 
Ficure 13-2 


Observe that if in the graph of an economy the numbering of nodes t 
and j is interchanged, then rows i, j and columns 2, j of the technology 
matrix are also interchanged. Thus an economy represented by a matrix A 
is completely decomposable if the matrix A can be cast into the form 
(13-45) by a series of interchanges of columns and corresponding rows. 

Now consider the economy whose graph is given in Fig, 13-2. This 
economy has a peculiar feature: Although the graph is connected, certain 
groups of industries are joined to other industries only by a one-way link, 
ie. they buy from these other industries, but do not sell to them. The 
industries in such an economy can be divided into groups so that every 
industry in group k directly or indirectly buys from every other industry 
in group k, but all industries in group, sell to industries in other groups 
only when the outside group has an index greater than k. The industries 
in Fig. 13-2 have been numbered so that they fall into three groups: 
(a) Industries 1, 2 in group 1 sell to each other, to industries 3, 4 in group 
2, and industry 5 in group 3; (b) industries 3, 4 sell only to each other 
and to industry 5; (c) industry 5 does not sell to any other industry. 
This economy is decomposable, but not completely decomposable. The 
technology matrix A (Fig. 13-2) has the form 


аур i2 413 G14 tis 


азр 022 0 0 а Ai Ais Ais 
А =) 0°" 10 Maas Page О ЦСО ТА 25 А3 
0, 0 аз ача а45 0 .0 Ass 


0 0 0 O0 ass 
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In a decomposable economy, it is not true that every industry buys 
directly or indirectly from every other industry. Hence the industries 
of a decomposable economy can be divided into groups such that each 
industry in a group buys directly or indirectly from all other industries 
in the group, but every industry in any group Ё can sell to an industry 
in another group only if the other group has an index greater than k. 
This means that if the industries are properly numbered, the technology 
matrix will have the triangular form. 


An Aiz- Ar 


0 А2: --Аз, 


A= : (13-46) 


0 0 --- An 


and the submatrices А, are square and cannot be written in the form 
(13-46). Note that a completely decomposable economy is a special case 
of the general form of the decomposable economy here described. If a 
decomposable economy whose matrix is given by (13-46) is not com- 
pletely decomposable, then it must be true that for every index j, there 
is either at least one submatrix Aj; = 0,7 < j, or at least one submatrix 
Aj; * Ofori > j. 

Note that if the industries in a decomposable economy are not num- 
bered properly, the matrix A will not have the form (13—46). In mathe- 
matical terms, an economy is decomposable if by interchanging columns 
and also interchanging the corresponding rows we can cast the technology 
matrix into the form (13-46), with the submatrices Ау; being square. 

The properties of economies which are decomposable and have tech- 
nology matrices of the form (13-46) are somewhat different from those 
of indecomposable economies. Suppose that b; in the bill of goods b is 
increased. Imagine that if A is partitioned as in (1346), ¢ falls in group Ё. 
Because every industry in group К directly or indirectly sells to all other 
industries in group k, every ту in group k must increase. However, since 
industries in group k do not buy from industries in groups with an index 
greater than k, the x; for industries in such groups will remain unchanged 
when b; is increased. The xj in some group 7 with an index smaller than 
k may or may not increase, depending on whether one or more industries 
in group k buy from one or more industries in group r. 

The above information gives us some indication of the nature of 
(I — A)~. It shows that if (I — А)! is partitioned in the same way 
as А, all elements below the main diagonal vanish, and (I — A)! also 
has a triangular structure. This follows because a change of b; in group Ё 
does not change ту if x; is in a group with an index greater than К. If 


To- 
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A,, are the submatrices of (I — A)7', then the elements of each Aj, are 
all positive because an increase in any one by of group k increases every 
ху of group К. We know that the elements of Am, s > r, will be non- 
negative, However, some Ar, may be null matrices. But if one element 
of A,, is positive, all elements will be positive for the following reason: 
If a change in some b; in group s has the effect of changing one x; in group г, 
it must cause a change in every x; in group F because every industry in 
group r buys from all other industries in group r. This requires that 
each clement of A,, be positive (why?). 


13-10 The closed Leontief model. In our previous diseussions, labor was 
treated as a primary factor, and the purchases by consumers were as- 
sumed to be given as the bill of goods or part of the bill of goods, Suppose 
that we wish to study the effects of a change in the level of foreign trade 
or in government spending. Such changes will induce a change in the 
amount of labor required in the system, which in turn will presumably 
cause a change in consumer demand. Hence consumer demand will not 
be independent of changes in the level of foreign trade or of government 
spending. 

To account for such changes we must relate consumer demand to the 
amount of labor used in the economy. This is done by treating labor ns 
if it were another industry. In earlier sections, we have assumed that if 
activity j is operated at a level ту, then aor; units of labor are used. There 
is no external demand for labor. Hence if xp is the total labor requirement, 
the labor balance equation reads 


zo — 22 oot, = 0 | 34m 
jml > 


We then assume that the consumer demand уо for product f is 
Yio = agr, S= ees (13-48) 


The consumer demand for each product i is directly proportional to the 
total amount. of labor employed. 

Let 6 be the bill of goods required for foreign trade and government 
orders. Write âa = [a,9,..., ano]. The system of equations which 


determine x, deinen 
Hog nol ЈЕ Jd (13-49) 
te 4 ; 
oli ^I 

This is a s; nl tions in n + 1 unknowns. The solution 


Se raat of each product to be made to meet con- 
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sumer demands and exogenous demands. It also determines the amount 
of labor to be employed. It сап be used ‘to study changes in Б because 
it accounts for the change that takes place in consumer demand as the 
total amount of employed labor is changed. 

Suppose now that we consider a completely closed system. No foreign 
trade exists, and government activities are either ignored or treated as 
an industry. Then (13-49) becomes 


| M CIA Е = 0. (13-50) 
-ù I—A]z 


In order that there be solutions other than the trivial solution, it is neces- 
sary that 
1 —а0 


—fio I-A 


When (13-51) holds, there will always be solutions with not all z; = 0. 
The solution to (13-50) does not determine the actual values of the vari- 
ables, It determines them up to a constant multiple since r(I — A) = n. 
This is to be expected for a closed system. Whatever the level of em- 
ployment, just enough will be produced to meet consumer demand. 

At this point, we must stop to examine the difference between (13-49) 
and (13-50). In order to solye (13-49), the matrix of the coefficients must 
be nonsingular. However, in (13-50) we require that the matrix of the 
coefficients must be singular. Clearly, these matrices which were written 
in identical form must be different. The difference lies in public con- 
sumption, In a closed economy, the public always consumes precisely 
what the economy produces, and it is impossible to meet an exogenous 
demand. For the economy described by (13-49) public consumption is 
always at least a little less than the total output. Hence if we move from 
(13-49) to (13-50), the vector & must change. This vector is all that 
needs to change. 

Equation (13-49) has another surprising feature, It says that if b = 0, 
then zo = 0, x = 0, that is, the economy does not produce anything if 
there are no exogenous demands. This is not realistic. The economy 
should be capable of operating at some level with not all z; = 0 to support 
consumer demand. The difficulty lies in the fact that a matrix is either 
singular or nonsingular. For Eq. (13-49) to behave realistically ав 6 
approaches 0, it is necessary to drop the assumption that the ao; are 
constants, and to permit the ару to change as b approaches 0 in such а 
way that the matrix of the coefficients becomes singular when b = 0. 

The above paragraph has shown that the ao; cannot be expected to 
remain constant over wider ranges of хо. In fact, it is not reasonable to 


= 0. (13-51) 
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expect any of the technological coefficients ay to remain constant over 
wide ranges of variation of ту. It is only for a relatively small range of 
variation of the variables that one may expect the assumption of constant 
technological coefficients to be even an approximate representation of 
reality, 


13-11 Practical uses for input-output models. A great deal of effort 
has been devoted to the task of representing the American economy by 
input-output models. Most of this work has been done by government. 
agencies, such as the Bureau of Labor Statistics and the Air Force, The 
model lends itself naturally to future planning; in fact, the main interest 
in the model derives from its suitability for planning purposes. For ex- 
ample, it is of interest to study employment levels and the gross produc- 
tion required to meet various specified bills of goods (consumption in- 
cluded). Of particular interest is the problem of deciding whether in times 
of war a bill of goods containing heavy demands by the military can be 
met with the available productive capacities and labor force, 

It is also useful to study whether serious unemployment may result 
from a change-over to a peace-time economy, 

The model for prices is also interesting. It is possible to study how 
an increase in the wage rate of steel workers will change prices throughout 
the whole economy if the assumption is made that the entire cost of the 
increase is passed on by increased prices, The general form of the price 
equations (13-29) is used in this case. It is not necessary to assume that 
the wage rate is the same throughout the whole economy or that the only 
value added is due to labor; it is only necessary to change the value added 
for one industry and to note the effects of this on the prices of all goods. 


13-12 Alternative activities. Our preceding discussions were based on 
the hypothesis that only a single industry (i.e., а single production process) 
makes a given product, Let us drop this assumption and imagine that 
several industries are capable of making the same product, We retain, 
however, the assumption that a given industry makes no more than one 
product. We shall denote by ey — aj, 4 = 1,... 173), the activity 
vectors for the industries which produce product j, and by 27 the levels 
at which these activities are operated, If b is the bill of goods, then the 
7; 2 0 must satisfy 


X Le = afr] = b. (13-52) 
РТТ ( 


Since there are now more variables than equations, the xj cannot be 
determined without some additional requirements. If we also wish to 
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minimize a linear objective function: 


n 
z= 27 cu, (13-53) 


then we have a linear programming problem which provides a way of 
selecting the industries which will operate at a positive level. 

There are n equations in the set of constraints (13-52), and therefore $ 
an optimal solution never need have more than n nonzero x}. If each 
b; > 0, then an optimal solution must contain at least one industry 
which turns out product j, 7 = 1,...,m. Since there are only n activities 
in the basis, an optimal solution will be such that one and only one in- 
dustry produces each produet. 

Now if all industries are legitimate, we expect that for any b > 0, 
every combination of n industries, with precisely one industry for each 
product, should yield a basic feasible solution to (13-52), i.e., should be 
able to produce the requisite amounts to meet the bill of goods. Let us 
suppose that we have some basic feasible solution which is not optimal. 
If the activity vector e; — aj is to enter the basis at the next iteration, 
it must replace the activity vector in the basis which produces product j. 
No computation is required to determine which vector should leave the 
basis. This in turn yields the interesting result that the set of optimal 
solutions is independent of the bill of goods b, since, in the simplex method, 
b has no influence on the vector to enter the basis; it only affects the vector 
to be removed. 

Here we have shown that the Leontief model can be generalized to 
include alternative production processes. The activities to be used are 
chosen by minimizing some linear objective function. 

Dantzig [1] has developed a short-cut technique for solving the linear 
programming problem (13-52), (13-53) for a decomposable economy. 
Let us assume that the technology matrix including the alternative 
activities has the form (13-46).* Dantzig calls such matrices block tri- 
angular, It will be assumed that all industries which produce a given 

good have the same structure so that if one industry which produces good j 
appears in Ату, all industries producing j appear in A1. 

We know that any basis matrix will contain one and only one activity 
vector for each good. Thus the basis matrices will also have a block- 
triangular form. From Section 13-9, we know that the inverse of any 
basis will also haye a block-triangular form so that we can write the 


* Here the Ар need not be square, however, since there can be alternative 
activities, 


$ 
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іпутве of any basis B as 


B'!-|. ESI (13-54) 


# ОУ 
A If B;; is the submatrix in B corresponding to Aj; in B^, the multiplication 
BB-! shows that Ay = Bj, i = 1,...,К. 
Let св contain the prices in the basis; Св will be partitioned in the 
same way as the rows of B7, so that 


cs = (Cai, Свз, ..:›Свь): (13-55) 


If in (13-46) Aj, is mı X m, then any activity vector in the first nı 
columns of A can be written in partitioned form as 


e; = aj = [fj 0,...,0). (13-56) 

Assuming that Ago is mg X m, we can write any activity vector in columns 
nı +1,..-, n +n in the partitioned form 

e; — aj = (6, B7 0,۰۰۰, 0). (13-57) 


The same can be done for the remaining columns of A. 
Now consider 2; — cj = caB (e; — aj) for any activity veetor in 
the first nı columns of А. Ву Eqs. (13-54) through (13-56), we have 


aj — cj = emBiifj — ©} (13-58) 


Note that z; — су is completely independent of any activities other 
than the first n, activities. Thus the optimal set of activities for the 
first mı columns of B can be determined independently of the remaining 
columns of B. No activity vector other than one of the first n, activity 
vectors in A will appear in the first m, columns of B. To carry out the 
computation, ВТ! is not needed; only Втр is required, 

Let us continue and see whether it is possible to optimize the columns 
mı + 1,..., mg of B in the same way. For an activity vector in columns 
nı + l, ...,mg of A, 


zj — cj = (св1Вїї, ©з1А1з + св2В22)10), gj] — c; (13-59) 
We evaluate A15 by considering the product BB-'. This gives 
By Ayo + BizA22 = 0, 
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or 


Ai = —BijBigBzz. (13-40) 
Substituting this result into (13-59), we obtain \ 
2) — cj = (свз — cpiBii Biz)Bz2g; — (cj — cniBiifj), \ 
or . Y 
£j — cj = êpaBz2 gj — 6j, (13-61) 
where E 
êna = (cma — cgiBiiBig); — 6; = cj — ca :Bri'fy. j 
Thus the columns mı + 1,..., ma of B can also be optimized inde- 


pendently of the remaining columns. We only need to compute By}. 
Equation (13-61) has the same form as (13-58) except that the prices 
must be modified. These prices depend on the first m, columns of B 
which we assume to be optimized. [ 

Continuing in this way, we obtain the optimal solution to the entire 
linear programming problem by block optimization. The amount of work 
needed to solve the problems by Dantzig's short-cut method is con- 
siderably less than that required by a direct application of the simplex 
method. 


13-13 Dynamic Leontief models. The Leontief economies studied thus 
far were static, and time did not appear. Leontief-type models have been 
generalized to include dynamic behavior. ‘These generalizations will be 
briefly discussed in this section. 

We shall be concerned with the state of the economy at different points 
in time. It will be assumed that some time period (a month or year, 
perhaps) is chosen, and the change in the economy from one time period 
to the next will be determined. As before, we assume that there are n 
industries. Each industry produces only one product and uses only а 
single process of production. 

It will be convenient to measure all quantities in monetary terms. Let 
yij(k) be the amount of good ? bought by industry j in time period k, 
and let x(k) be the gross production of good j in time period k. Then 
Leontief assumes that the inputs are related to the outputs by 


ШК) = аул, ' all i j, k, (13-62) 


where aj; is a constant which does not depend on the time period under 
consideration. Y 

The dynamic model also considers capital stocks. Let 8;;(k) be the capital 
stock of good i held by industry j at the beginning of time- period k. 
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Leoitief assumes that the в, (К), = 1, . . .., n, needed to support the pro- 
ducion of j will be proportional to the amount of good j produced in 
tine period k, i.e., 


salk) = djz(K) ^ alij k, (13-63) 


where the dj; > 0 are independent of the time period (of course, some of 
the dj may be zero). The total capital stock s(k) of good € held by the 
ейге economy (exclusive of consumers and other exogenous users of 
Í product i) is 


s((k) = 5; su) = Y dirit), f= J,...,m (13-04) 
i ie 


Then As(k) = s,(k + 1) — 8:00), which is the change in the capital 
stock of product i from the beginning of time period k to the end of this 
time period, must be produced by industry i in time period k. Leontief 
assumes that As;(k) > 0, i = 1,...) n, 80 that for the whole economy, 
there is never disinvestment in any capital stock, i.e., there is no depletion 
of productive capacity. Some of the individual sj(k) may decrease, but 
&,(k) can never decrease. 

Take bilk) to be the exogenous demand for product # in time period k. 
Leontief assumes that the gross production for each industry in each 
time period exactly meets the total demands for the product, i.e., 


0 = So аю + аю b Em Lym (18-60) 
jul 


and 


ask) = È dd D ар #=1,...›® 08-00) 
j=l 


Now define 
А = flay, р = idal, Ба (0100)... „о, 
ж i, scl tem In(B ees tS OD 
In matrix form, the dynamic Leontief model can then be summarized as 
(I— A) — Duy + px, = bi, (13-67) 
в, = Dx. (13-68) 


The matrix D is assumed to be nonsingular. ‘The set of equations (13-67) 
is a set of n linear first-order difference equations. 
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If we are given the b; for all k, an initial gross-production vect 
and initial stocks во = Dxo, then the behavior of the entire econo) 
determined for all future times since 


хн = Xe + DC — A) — D^!b;. (13-49) 


Tt does not follow that the economy, starting from an arbitrary xo, 
always meet the bills of goods b; and also maintain As; > 0. In id 
it may be necessary for both As; and s; to become negative in order t 
satisfy the two requirements. Leontief avoided this problem by making 
the d;; nonlinear, ie., the di; changed values when any Az,(k) = 
vj(k--1) — 2;(k) changed sign. This introduced a new type of behavior ё 
into the system, which is sometimes referred to as the relaxation phe- 
nomenon. A discussion of this subject would take us too far afield. 
However, the reader can find a study of the two-variable case in an article 
by Georgescu-Roegen [3]. 

Leontief based his development on a continuous change in time rather 
than on diserete time periods. Consequently, he dealt with production 
rates rather than amounts produced in a given time period, and obtained 
a set of first-order differential equations rather than difference equations. 
The differential-equation and difference-equation models do have the same 
qualitative behavior, 

We can easily convert the deterministic, dynamic Leontief model to a 
form which requires that a linear programming problem be solved at each 
time step. Suppose that we replace (13-65) by 


X > Ах, + As, + by, all k. (13-70) 


Xo, 
is 


The strict inequality can hold for one or more industries only if their 
output is so large that it cannot be used. We also write (13-64) as 


Sk > Dx, all k. (13-71) 


The inequality simply means that the capital stock may not be used at 
full capacity in the production for the given period, i.e., excess capacity 
can exist, 


If we are given s, and wish to determine x, and As, + b; which maxi- 


mize some linear form 2 — c (As, + bj), then we have the linear pro- 
gramming problem 


(I — A)x. — (As, + bj) > 0, 
Dx, < Sp, 
pi fe (13-72) 
x, > 0, Ask + by > 0, 


max 2 = с(Аѕь + by). 
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the ojtimal solution to (13-72) may or may not satisfy As, > 0. How- 
everjthere will be at least one b (perhaps b = 0) for which As; > 0. 
In siving this linear programming problem, we maximize the growth 


of е economy in some way. The vector c determines the manner in 
the growth is being maximized. 
addition to the above alternatives, there are many other ways of 
lining dynamic Leontief-type models. We shall terminate our presen- 
ion by considering a model suggested by Wagner [5]. We assume that 
ndustries produce m products. More than a single industry may pro- 
ce a given product. As usual, however, no industry produces more 
an a single product. Let A be the technology matrix for the economy. 
t will be assumed to be independent of time so that there is no techno- 
ogical improvement. Take x; to be the gross production vector for period 
/ f, and let s be a vector representing the stocks of the various products 
available at the end of time period k, ‘These stocks represent quantities 
produced, but not used, by the end of time period k. We do not say who 
holds these stocks. The bill of goods for period k will be written bx. We 
shall also assume that limits are imposed on the productive capacity, 
that is, x, < qi. However, дь can change from period to period because 
some of the gross production is used to build more productive capacity. 
Assume that an increase of the productive capacity of industry j by an 
amount ту requires an amount d;jr; of product i. Let D = lldi;ll, and 
ry be a vector containing the increases in productive capacity built in 
period k. Imagine that this increase in productive capacity cannot be 
used until period k + 1. 
If qo represents the initial productive capacities and bj, ry, Sy, 8к—1 
are given for time period k, then x, > 0 must satisfy 


о! 


(I— A)x, = sp — Sii + Dri + by, (13-73) 
ki 

Xk X qo + MN rj. (13-74) 
j=l 


It may or may not be true that the хь determined from (13-73) satisfies 
(13-74). 

Suppose that we are interested only in a total of n time periods and 
that we wish to maximize z = eqn = 190 + 2-72) rj, that is, some 
linear combination of the productive capacities for the final time period. 
At the same time, we must satisfy each bill of goods by, k = 1,...,т, 
if this is possible. Assume that we are given the initial productive ca- 
pacities and inventories qo, so, respectively. Then the problem is to 
determine the xy, ry, k = 1,...,n, which maximize z. This means that, 


508 LINEAR PROGRAMMING AND ECONOMIC THEORY 


we must solve the linear programming problem 


(I — A)x, = sy — S, 1 + Dr; + by, k=l, n, 
k— 
х, < 4 DO rj 
ja 
x20 r 20, 


n=l 
Шен: es Р di 


j=1 


Since upper bounds limit the capacities, it is quite possible that, in a 
optimal solution, more than a single industry will produce any given — 
product. This may be true in one time period or all time periods. Y 

It is much more difficult to represent the dynamic behavior of an 
economy than it is to study the static behavior over a relatively short 
time period. The attempts made to generalize Leontief models so that they 
will cover dynamic behavior illustrate this difficulty. Many new problems 
are encountered, and many more assumptions are needed, ‘The models 
discussed here do not present a very adequate description of the real 
world since they omit consideration of many important factors, such as 
technological improvement, etc. However, as varieties of dynamic models, 
they are of interest in theoretical economics. 
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PROBLEMS 


13-. Consider a firm which produces a single good and operates under pure 
comptition. If g is the production rate, the firm incurs costs at the rate of 


C(q) = 0.003(q — 50)3 + 25. 


Thmarket price for the firm's product isp = 10. What production rate maxi- 
mies the firm's profit rate? What is the maximum profit rate? On a single 
grph, plot the marginal and average cost curves as well as the marginal revenue 
cuves. On another graph, plot R(g) = pq, C(q), Ш(д) = Е — C. What is 
fle equation for the firm's supply curve? 

13-2. A manufacturer of a single good buys and sells in competitive markets. 
There are two factors of production. If zı, x2 are the rates at which these fac- 
fors are used, the production rate q is related to zı, 22 by the production function 


q = зїї. 
Costs are incurred at the rate 
C = wizi + were + k. 


If p is the market price of the good, find the production rate which maximizes 
the profit rate. Also, find the demand functions for the factors of production. 
13-3. The demand curve for a good produced by a monopolist is 


q = —200p + 1000, 


where p is the price and q the rate of demand at this price. If the monopolist 
produces at a rate q, he incurs costs at a rate of 


€ = 10-44 — 100)3 + 250. 


Find the production rate which maximizes the profit rate. Also, find the price 
which should be charged to maximize the profit rate. Plot the profit, revenue, 
and cost curves as a function of q. If ĝ is the price which maximizes the profit 
rate, what production rate would maximize the profit rate if the manufacturer 
sold in a competitive market with market price P? What would be the profit 
rate? How do you explain the result obtained? 

13-4. Assume that a firm which makes joint products buys and sells under 
pure competition. Show how the formulation given in Section 13-2 for joint 
produets accounts for changes in some process variable, such as temperature or 
pressure, which can change the mix of the joint products. Hint: An increase in 
the temperature of some unit may require a larger electrical load on the heating 
Ds or an increase in the pressure on a unit may require mor- work from the 

umps. 

13-5. Consider the set of equations (13-5). Write these as 


fízn..., 2. Wi- -er Wn, p) = 0, = 1,...,п. 
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Choose a particular ir, кау tx. Show how to compute д/д, i = 1)... n, 
Under what condition is it possible to obtain a unique solution for thesepartial 
derivatives? \ 
13-6, In Section 13-3, we have defined an activity as a process of prod\ction 
yielding only a single product. Show how to generalize the analysis so tht an 


activity may involve the production of several products, i.c., joint prodets. 
Hint: Let the variable № be the level at which the activity a; is operated. ‘hen 
Aj, gives the amount of the resources used when a; is operated at the leve Aj 


Let byy be the amount of good £ produced when the activity is operated at 
level Ay. What price should be used for А? ۴ 

13-7, A farmer can use three activities at any level: Tho first produces 1090 
hogs by means of 30 man-months of labor, 50 acres of land, and 200 tons of cor. 
The second produces 100 tons of corn, using 20 man-months of labor and 10) 
acres of land. The third produces 500 hogs and 200 tons of corn, using 40 mam 
months of labor and 130 acres of land. Corn сап be bought or sold at $100/ton, 
and hogs ean be sold at $30 each. The farmer has available 50 man-months of 
labor and 150 acres of land. At what level should each activity be operated in 
order to maximize profits? 

13-8. Consider an economy consisting of two industries whose technology 
matrix in terms of natural units is 


` 
a - [005-025]. 
0.15 020 


How much should each industry produce to meot the bill of goods b = (20, 50]? 

13-9, Given the technology matrix of Problem 13-8, how much of good 1 
must be used to produce 80 units of good 1 and 10 units of good 2? How much of 
good 2 is needed? 

13-10. For the technology matrix of Problem 13-8, find the prices of the 
goods if the values added per unit are 0.50 for good 1 and 0.32 for good 2. An 
increase in wages increases the value added per unit in industry 1 to 0.60. If 
the entire cost in passed on, what are the new prices for both goods? 

13-11. Use the prices determined in the first part of Problem 13-10 to con- 
‘vert the technology matrix of Problom 13-8 to monctary units, 

13-12. A hypothetical economy consists of three industries: agriculture, 
autos, and chemicals, The distribution of the output of each industry during 
the past year is shown in Table 13-2. Compute the technology matrix A in 
natural units and (I — A) =1, 

13-13. For the economy described in Problem 13-12, assume that labor is 
the only primary factor and that the wage rate is $1.50 per hour. The labor 
hours are: 5 X 10% per million pounds of chemicals, 8.2 X 10° per million autos, 
арі 6 X 10° per million carloads of agricultural products. Compute the prices 
which prevail in the economy if profits are neglected. Compute the technology 
matrix in monetary units. 

13-14. For the economy presented in Problem 13-12, find how the output 
will change if exports of chemicals increase from 1 to 2 million pounds per year. 


—9 


(millions of Ibe) 


Autos 
(millions of vehicles) 


Agriculture 
(millions of earloada) 


13-15, Write the form of the technology matrix ран 9h ини 
form) for an economy which ean be represented by the graph in Fig. 13-3. 
* 13-16, A hypothetical economy consists of three industries: (1) agriculture, 
(2) metal products, (3) chemicals, For each industry there are three processes 

of production, The theen possible. sn AEA ч ИЙ be died DF 
Ai, аз, аз. In monetary unita, these activity vectors are 


a: = (01,02,02) аз = [02,03,01], аз = (0.15, 0.25, 0.30]. 
The first component of each of these vectors refers to agriculture and the third 
to chemicals, The three activity: veetorn МИМИ products vil M ИЙНИ by 
M4, аз, аб; they are 

as = (0.05, 0.3, 0.3), аз = (0.03,04, 0.2), в = (0,07, 0.25, 0.35), 
The three activity vectors for chemicals will be denoted by ат, as, as; they are 


a7 = (0.25, 0.05, 0,15), aa = (0.30, 0.04, 0,13), ap = 10.10, 0.08, 0.25), 
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оиа 


FiaunE 13-4 


Let т; be the level of activity j. Activity 4 has an upper bound of 4 X 109, and 
activity 7 has an upper bound of 3 X 109. The labor requirements uj per unit 
level of each activity are ші = 0.45, u2 = 0.25, из = 0.30, u4 = 0.30, us = 
0.35, ug = 0.45, ит = 0.42, из = 0.50, ug = 0.55. Determine the level at 
which each activity should be operated in order to minimize labor requirements 
,and to meet exaetly the exogenous bill of goods: 


b = [18 X 106, 10 X 108, 6 x 108]. 


13-17. Write down the dual of a gasoline-blending problem of the type dis- 
cussed in Chapter 12. Can you provide an economic interpretation for the dual 
variables? 

13-18. Consider the dual of the refinery routing problem discussed in Chapter 
12. Can you provide an economic interpretation for all (or any) of the dual 
variables? 

13-19. Consider an economy represented by the graph of Fig. 13-4. Write 
down the technology matrix for this economy in its decomposed form. Write 
down the form of (I — A) 71, indicating clearly which elements are zero, Evalu- 
ate explicitly (I — A)-! in terms of the elements of A for an economy whose 
graph is shown in Fig. 13-5. 

13-20. Give the form of the technology matrix for an economy: whose graph 
is shown in Fig. 13-6. Compute (I — A)-! in terms of the elements of A. 


TTN 


Ficure 13-5 Figure 13-6 
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13-21. Show that for any linear programming problem Ax = b, x > 0, 
max г = cx, the dual variable wy can be interpreted as 02max/0b,, provided that 
b, can be varied independently of the other b/s. For the transportation problem, 
is it possible to change an a; without changing a b;? Hint: Da; = Zb,. 

13-22. For the formulation of the transportation problem used in Chapter 9, 
where the dual variables were unrestricted in sign, we noted that any one of the 
dual variables could be assigned arbitrarily and the others would be uniquely 
determined. Show that for the formulation of the transportation problem given 
by Eq. (13-19), where the dual variables are non-negative, it is not possible to 
assign arbitrarily a value to one of the dual variables. Why is this so? Hint: 
In Eq. (13-19) there are surplus variables which must be added. Will one of 
the surplus vectors appear in an optimal basis? 

13-23. Consider a simple economy consisting of three industries: (1) metals, 
(2) electronics, (3) missiles. It is desired to study the dynamic behavior of this 
economy over a five-year period, the object being to maximize the production 
of missiles. To increase missile production, it may also be necessary to increase 
the capacities of the metals and electronics industries. The natural units for 
measuring production are number of missiles, tons of metal, and “equivalent 
computers” in the electronics industry. In natural units, the technological co- 
efficients for the economy are given in the following table. The last row of the 
table gives the labor coefficients ao; in man-years. 


Electronics 


Missiles 
Electronics 
Metals 
Labor 


The capacities of each industry are measured in the same units as production. 
The coefficients d;; which give the units of 1 required for increasing the capacity 
of industry j by one unit are given in the following table: 


Metals 


Missiles 0 


Electronies 
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Any increased capacity obtained in one period cannot be used until the beginning 
of the next. The initial capacities of the industries in their natural units are 
(1) missiles: 100, (2) electronics: 4000, (3) metals: 100,000. One million man- 
years of labor are available in the first year. The labor force is expected to in- 
crease at the rate of 10% per year. The exogenous demand for electronics in 
the first year is 3000 and for metals 60,000. The exogenous demand for elec- 
tronics is expected to increase at the rate of 10% per year and for metals at the 
rate of 5% per year. Assume that electronics and metals can be inventoried 
and that the initial stocks of metals and electronics are respectively 5000 and 
100. Set up on a large sheet of paper the linear programming problem whose 
solution will maximize the output of missiles over the next five years while at 
the same time meeting the exogenous demands. If the reader has access to a 
digital computer, he will find it instructive to solve the above problem and do a 
parameter study determining how the missile production depends on the initial 
electronics capacity. 
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